WO2022156520A1 - 一种云路协同的自动驾驶模型训练、调取方法及*** - Google Patents

一种云路协同的自动驾驶模型训练、调取方法及*** Download PDF

Info

Publication number
WO2022156520A1
WO2022156520A1 PCT/CN2021/144059 CN2021144059W WO2022156520A1 WO 2022156520 A1 WO2022156520 A1 WO 2022156520A1 CN 2021144059 W CN2021144059 W CN 2021144059W WO 2022156520 A1 WO2022156520 A1 WO 2022156520A1
Authority
WO
WIPO (PCT)
Prior art keywords
automatic driving
vehicle
driving model
cloud
data
Prior art date
Application number
PCT/CN2021/144059
Other languages
English (en)
French (fr)
Inventor
包哈达
王磊
冉雪峰
潘晏涛
Original Assignee
国汽智控(北京)科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 国汽智控(北京)科技有限公司 filed Critical 国汽智控(北京)科技有限公司
Publication of WO2022156520A1 publication Critical patent/WO2022156520A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G06F9/4881—Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/60—Software deployment
    • G06F8/65—Updates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals

Definitions

  • the present application relates to the technical field of automatic driving, and in particular to a cloud-road collaborative automatic driving model training and retrieval method and system.
  • the technical problem to be solved by this application is to overcome the defect that the existing automatic driving technology cannot perform individualized training and optimization for factors such as individuals or road sections, so as to provide a cloud-road collaborative automatic driving model training , and retrieval method and system.
  • an embodiment of the present application provides a cloud-road collaborative automatic driving model training method, including the following steps:
  • the vehicle-mounted sensing equipment sends the collected vehicle status data to the roadside equipment in real time through the IoV protocol;
  • the roadside equipment merges the newly uploaded data from the vehicle-mounted sensing equipment with the previously stored data into a data set, and preprocesses the data set;
  • the on-board controller controls the preprocessed data set to be trained and updated on the roadside equipment for automatic driving model training, or controls the transmission of the preprocessed data set to the cloud server for automatic driving model training update;
  • the updated model is uploaded to the cloud server;
  • the preprocessed data set is transmitted to the cloud server through Ethernet, and the automatic driving model is trained together with the data set in the cloud server, and the updated automatic driving model is fed back to the roadside equipment.
  • the data newly uploaded by the in-vehicle sensing device includes: the driver's intervention data on the driving behavior formed under the guidance of an existing automatic driving model.
  • the preprocessing process includes performing ground truth labeling and data cleaning according to the driver's intervention data.
  • spatial downsampling, spatial feature extraction and spatial information reconstruction are sequentially performed.
  • an embodiment of the present application provides a cloud-road collaborative automatic driving model retrieval method, including the following steps:
  • the automatic driving model in the roadside equipment is called to guide the vehicle's automatic driving behavior.
  • the automatic driving behavior in the cloud server is called.
  • the model guides the automatic driving behavior of the vehicle, and the automatic driving model is obtained based on the automatic driving model training method of cloud-road collaboration described in the first aspect.
  • the preset range is determined according to the range and frequency of the historical trajectory of the vehicle.
  • an embodiment of the present application provides a cloud-road collaborative automatic driving model training system, including: a vehicle-mounted sensing device, a vehicle-mounted controller, and a cloud server, wherein:
  • In-vehicle sensing equipment which is used to send the collected vehicle status data to roadside equipment in real time through the Internet of Vehicles protocol;
  • the roadside equipment is used to combine the newly uploaded data of the vehicle-mounted sensor equipment with the previously stored data into a data set, and to preprocess the data set;
  • the vehicle-mounted controller is used to control the automatic driving model training and updating of the pre-processed data set on the roadside equipment, or to control the transmission of the pre-processed data set to the cloud server for automatic driving model training and updating;
  • the roadside equipment is also used to train and update the automatic driving model on the roadside equipment based on the preprocessed data set, and upload the updated model to the cloud server;
  • the cloud server is used to obtain the data set preprocessed by the roadside equipment and train the automatic driving model with the stored data set, and feed back the updated automatic driving model to the roadside equipment.
  • the application embodiments provide a cloud-road collaborative automatic driving model retrieval system, including:
  • the vehicle location information acquisition module is used to acquire the vehicle location or the navigation path of the vehicle;
  • the automatic driving model retrieval module is used to fetch the automatic driving model in the roadside equipment to guide the automatic driving behavior of the vehicle when the vehicle positioning or the navigation path of the vehicle is within the preset range, and when it is not within the preset range , calling the automatic driving model in the cloud server to guide the vehicle on automatic driving behavior, and the automatic driving model is obtained based on the automatic driving model training method of cloud-road collaboration described in the first aspect.
  • the embodiments of the present application provide a computer-readable storage medium, where the computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the second aspect of the embodiments of the present application.
  • the cloud-road collaborative automatic driving model retrieval method is used to cause the computer to execute the second aspect of the embodiments of the present application.
  • an embodiment of the present application provides a terminal, including: a memory and a processor, the memory and the processor are connected in communication with each other, the memory stores computer instructions, and the processor executes the The computer instructions are used to execute the cloud-road collaborative automatic driving model retrieval method described in the second aspect of the embodiments of the present application.
  • the cloud-road collaborative automatic driving model training method and system integrates vehicle status collected by on-board sensing equipment and road data stored by roadside equipment, which can be carried out based on factors such as individuals or road sections.
  • the roadside equipment can receive vehicle status data through the roadside Internet of Vehicles protocol and update the model with its own local data set, which can save the bandwidth of data transmission, and at the same time integrate with the powerful computing of the cloud server to improve the Computational efficiency
  • the training process combined with the driver's intervention information enables the in-vehicle system to be adaptively trained, and the recognition effect of the automatic driving model can be continuously improved without human intervention. It has strong practicability and can improve the driving experience.
  • the cloud-road collaborative automatic driving model retrieval method and system provided by the embodiment of the present application, according to the vehicle's positioning and navigation path combined with the preset range determined by the historical trajectory range and historical trajectory frequency, the selection of priority retrieval is more personalized.
  • the automatic driving model of the roadside equipment terminal, or the automatic driving model of the cloud server with more generalization ability is preferentially called for automatic driving guidance, which can achieve a more accurate automatic driving recognition effect and give users a better driving experience.
  • FIG. 1 is a flowchart of an example of a cloud-road collaborative automatic driving model training method provided in an embodiment of the application
  • FIG. 2 is a flowchart of an example of a method for retrieving an automatic driving model of cloud-road collaboration in an embodiment of the present application
  • FIG. 3 is a module composition diagram of a specific example of the cloud-road collaborative automatic driving model retrieval system provided in the embodiment of the application;
  • FIG. 4 is a composition diagram of a specific example of a terminal provided by an embodiment of the present application.
  • the embodiment of the present application provides a cloud-road collaborative automatic driving model training method, as shown in FIG. 1 , the method includes the following steps:
  • Step S11 The vehicle-mounted sensing device sends the collected vehicle state data to the roadside device in real time through the vehicle networking protocol.
  • Step S12 The roadside equipment merges the data newly uploaded by the vehicle-mounted sensing equipment and the previously stored data into a data set, and preprocesses the data set.
  • Step S13 the on-board controller controls the preprocessed data set to be trained and updated on the roadside equipment for automatic driving model training, or controls the transmission of the preprocessed data set to the cloud server for automatic driving model training update;
  • Step S141 when the automatic driving model is trained and updated on the roadside device based on the preprocessed data set, upload the updated model to the cloud server;
  • Step S142 When the cloud server performs training and updating of the automatic driving model, the preprocessed data set is transmitted to the cloud server through Ethernet, the automatic driving model is trained together with the data set in the cloud server, and the updated automatic driving model is fed back to roadside equipment.
  • the cloud-road collaborative automatic driving model training method fuses the vehicle status collected by the on-board sensor equipment and the road data stored by the roadside equipment. Personalized training and optimization, and the recognition effect of machine learning largely depends on the amount and diversity of data. For automotive terminals, the condition that restricts the amount of data is network bandwidth.
  • the data is passed through the roadside
  • the device and the car's Internet of Vehicles protocol (such as the v2i network protocol) are transmitted to provide transmission efficiency, and then use the computing nodes deployed on the roadside to combine the data uploaded by the car and the data stored on the roadside into a data set. Data preprocessing.
  • the roadside equipment preprocesses the data in the embodiment of the present application, the amount of data will be greatly reduced, and the data can be transmitted to the cloud server for task scheduling more quickly, and then a new model can be trained together with the data set in the cloud, and then the model can be Feedback to the roadside equipment; if the computing nodes owned by the roadside equipment are powerful enough to undertake part of the training work of the model, the roadside equipment can receive vehicle status data through the roadside Internet of Vehicles protocol and combine it with its own local data set. Model update can save the bandwidth of data transmission, and only need to transmit the interactive information of the task processing results to the cloud server.
  • the data newly uploaded by the vehicle-mounted sensing device provided by the embodiment of the present application includes: the driver's intervention data on the driving behavior formed under the guidance of the existing automatic driving model; the preprocessing process includes the real value labeling and Data cleaning.
  • the factor of whether the vehicle-mounted sensing device and the driver are involved is used to judge the true value of the collected data. For example, other sensors have identified obstacles, but the data collected by the image device has not been identified. The corresponding It is considered that the image recognition model is not good for the recognition of the object, and the raw data of the image can be marked in combination with the sensor data, and the data can be transmitted to the automatic driving model for further training. When a driver intervenes, the driver's judgment is taken as the true value, the error information of some sensor data is marked, and then the error data is processed to improve the recognition accuracy of the automatic driving model.
  • the dynamic loading technology can be used to write a general interface, so that the on-board system can freely load and unload the module in a pluggable manner, cooperate with the life cycle management, and the remote download platform can be The system resources are allocated and managed through the life cycle, so that the application program can be expanded and maintained more easily.
  • the recognition module and automatic driving model can be reloaded by dynamic loading.
  • an embodiment of the present application also provides a cloud-road collaborative automatic driving model training system, including: on-board sensing equipment, on-board controller and cloud server, wherein the on-board sensing equipment is used to collect collected vehicles The status data is sent to the roadside equipment in real time through the Internet of Vehicles protocol; the roadside equipment is used to combine the newly uploaded data from the on-board sensing equipment with the previously stored data into a data set, and preprocess the data set; on-board control It is used to control the automatic driving model training and update of the preprocessed data set on the roadside equipment, or to control the transmission of the preprocessed data set to the cloud server for automatic driving model training and update; the roadside equipment is also used for automatic driving model training and update based on The preprocessed data set is used to train and update the automatic driving model on the roadside equipment, and the updated model is uploaded to the cloud server; the cloud server is used to obtain the preprocessed data set of the roadside equipment and store it in the The datasets are used to
  • the cloud-road collaborative automatic driving model training method and system integrate vehicle status collected by on-board sensing equipment and road data stored by roadside equipment, and can be personalized for factors such as individuals or road sections.
  • the roadside equipment can receive the vehicle status data through the roadside vehicle networking protocol and update the model with its own local data set, which can save the bandwidth of data transmission, and at the same time integrate with the powerful computing of the cloud server to improve the calculation.
  • Efficiency the training process combined with the driver's intervention information enables the in-vehicle system to be adaptively trained, and the recognition effect of the automatic driving model can be continuously improved without human intervention. It is highly practical and can improve the driving experience.
  • the embodiment of the present application provides a cloud-road collaborative automatic driving model retrieval method, as shown in FIG. 2 , including:
  • Step S21 Obtain vehicle positioning or a navigation path of the vehicle.
  • Step S22 When the vehicle positioning or the navigation path of the vehicle is within the preset range, the automatic driving model in the roadside equipment is called to guide the vehicle on automatic driving behavior, and when it is not within the preset range, the automatic driving behavior in the cloud server is called The autonomous driving model of the vehicle guides the autonomous driving behavior of the vehicle.
  • the preset range is determined according to the historical trajectory range and historical trajectory frequency of the vehicle.
  • the vehicle positioning is a route that must be passed between the owner's home location and work location, or a route that is often active on weekends.
  • the automatic driving model in the roadside equipment is retrieved. If it is not a route where the car owner often travels, such as self-driving tours in other cities, the automatic driving model of the roadside device terminal is retrieved.
  • the cloud server Since the roadside equipment mainly collects and preprocesses data within the specific trajectory range of the target vehicle, and the cloud server receives the models or preprocessed data uploaded by all roadside equipment, the cloud server is more efficient than the automatic driving model stored in the roadside equipment. It is generalizable, and the automatic driving model stored by roadside equipment is more personalized than the cloud server. Therefore, when the vehicle implements the automatic driving mode, it is necessary to first obtain the vehicle positioning or the vehicle's navigation path, which is the range of the vehicle's frequent activities.
  • the embodiment of the present application further provides a cloud-road collaborative automatic driving model retrieving system, as shown in FIG. 3 , including:
  • the vehicle location information acquisition module 21 is used to acquire vehicle positioning or the navigation path of the vehicle; this module is a functional module integrating positioning and navigation functions, and this module executes the method described in step S21 in Embodiment 1, which will not be repeated here. .
  • the automatic driving model fetching module 22 is used for fetching the automatic driving model in the roadside equipment to guide the automatic driving behavior of the vehicle when the vehicle positioning or the navigation path of the vehicle is within the preset range, and when the vehicle is not within the preset range When , the automatic driving model in the cloud server is called to guide the automatic driving behavior of the vehicle.
  • This module executes the method described in step S22 in Embodiment 1, and details are not repeated here.
  • the cloud-road collaborative automatic driving model retrieval method and system selects the priority to retrieve a more personalized road according to the vehicle's positioning and navigation path combined with the preset range determined by the historical trajectory range and historical trajectory frequency.
  • the automatic driving model of the side device terminal is still the priority to call the automatic driving model of the cloud server with more generalization ability for automatic driving guidance, which can achieve a more accurate automatic driving recognition effect and give users a better driving experience.
  • the device may include a processor 51 and a memory 52 , where the processor 51 and the memory 52 may be connected through a bus or in other ways.
  • FIG. 4 takes the connection through a bus as an example.
  • the memory 52 can be used to store non-transitory software programs, non-transitory computer-executable programs and modules, such as corresponding program instructions/modules in the embodiments of the present application.
  • the processor 51 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 52, that is, to realize the automatic driving model of cloud-road coordination in the above method embodiment 2 call method.
  • the memory 52 may include a storage program area and a storage data area, wherein the storage program area may store an operating system and an application program required by at least one function; the storage data area may store data created by the processor 51 and the like. Additionally, memory 52 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 52 may optionally include memory located remotely from processor 51 , which may be connected to processor 51 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, intranets, mobile communication networks, and combinations thereof.
  • One or more modules are stored in the memory 52, and when executed by the processor 51, execute the method for retrieving an automatic driving model for cloud-road collaboration in Embodiment 2.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive) , abbreviation: HDD) or solid-state drive (Solid-State Drive, SSD), etc.; the storage medium may also include a combination of the above-mentioned types of memories.

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
  • Traffic Control Systems (AREA)

Abstract

云路协同的自动驾驶模型训练、调取方法及***，模型训练方法及***将车载传感设备与路侧设备、云端服务器进行整合，同时结合驾驶员的介入信息使车载***能够自适应进行训练，可以针对个人或路段等因素进行个性化的训练与优化，不需要人为介入即可持续提升自动驾驶模型的识别效果；模型调取方法及***根据车辆的定位及导航路径结合历史轨迹范围及历史轨迹频率确定的预设范围，选择优先调取更具个性化的路侧设备或更具泛化能力的云端服务器的自动驾驶模型进行自动驾驶指导，可以达到更加精确的自动驾驶识别效果，给用户更好的驾驶体验。

Description

一种云路协同的自动驾驶模型训练、调取方法及*** 技术领域
本申请涉及自动驾驶技术领域，具体涉及一种云路协同的自动驾驶模型训练、调取方法及***。
背景技术
在当前的自动驾驶环境中，深度学习在感知与决策部分承担着非常重要的职责，其中对于深度学习的模型的训练决定着识别效果与决策效果，进而影响自动驾驶的整体体验。传统的模型训练方法，是通过开发人员采集数据，在开发阶段进行数据清洗，数据标注，模型训练，训练好模型后部署到终端汽车上。这个流程中一旦车量产出厂，模型的更新就只能依赖OTA手段进行更新，并不能针对个人或者路段等因素进行个性化的训练与优化。现有的技术中，对于数据个性化的使用都没有被重视起来，对于自动驾驶而言，用户的场景通常是比较个人化的，对于经常行驶的路段，控制与感知的算法精度如果不能有明显的提升那么实际的驾驶体验不会有提升。
发明内容
因此，本申请要解决的技术问题在于克服现有自动驾驶技术中在不能很好的针对个人或者路段等因素进行个性化的训练与优化的缺陷，从而提供一种云路协同的自动驾驶模型训练、调取方法及***。
为达到上述目的，本申请提供如下技术方案：
第一方面，本申请实施例提供一种云路协同的自动驾驶模型训练方法，包括如下步骤：
车载传感设备将采集的车辆状态数据通过车联网协议实时发送至路侧设备；
路侧设备对车载传感设备新上传的数据与之前已经储存的数据合并为数据集，并对该数据集进行预处理；
车载控制器控制将预处理后的数据集在路侧设备进行自动驾驶模型训练更新，或控制将预处理后的数据集传输至云端服务器进行自动驾驶模型训练更新；
当基于预处理后的数据集在路侧设备对自动驾驶模型进行训练更新时，将更新后的模型上传至云端服务器；
当云端服务器进行自动驾驶模型训练更新时，将预处理后的数据集通过以太网传输至云端服务器，与云端服务器中的数据集一起训练自动驾驶模型，将更新后的自动驾驶模型反馈至路侧设备。
在一实施例中，车载传感设备新上传的数据包括：驾驶员对之前已有自动驾驶模型指导下形成行驶行为的介入数据。
在一实施例中，预处理的过程包括根据驾驶员的介入数据进行真值标注及数据清洗。
在一实施例中，将低分辨的高光谱图像输入到空间重构网络后，依次进行空间下采样、空间特征提取和空间信息重构处理。
第二方面，本申请实施例提供一种云路协同的自动驾驶模型调取方法，包括以下步骤：
获取车辆定位或车辆的导航路径；
当车辆定位或车辆的导航路径在预设范围时，调取路侧设备中的自动驾驶模型对车辆进行自动驾驶行为指导，当不在所述预设范围内时，调取云端服务器中的自动驾驶模型对车辆进行自动驾驶行为指导，所述自动驾驶模型基于第一方面所述的云路协同的自动驾驶模型训练方法得到。
在一实施例中，所述预设范围根据车辆的历史轨迹范围及历史轨迹频率确定。
第三方面，本申请实施例提供一种云路协同的自动驾驶模型训练***，包括：车载传感设备、车载控制器及云端服务器，其中：
车载传感设备，用于将采集的车辆状态数据通过车联网协议实时发送至路侧设备；
路侧设备，用于对车载传感设备新上传的数据与之前已经储存的数据合并为数据集，并对该数据集进行预处理；
车载控制器，用于控制将预处理后的数据集在路侧设备进行自动驾驶模型训练更新，或控制将预处理后的数据集传输至云端服务器进行自动驾驶模型训 练更新；
路侧设备还用于基于预处理后的数据集在路侧设备对自动驾驶模型进行训练更新，并将更新后的模型上传至云端服务器；
云端服务器，用于获取路侧设备预处理后的数据集并将其已存储的数据集一起训练自动驾驶模型，将更新后的自动驾驶模型反馈至路侧设备。
第四方面，申请实施例提供一种云路协同的自动驾驶模型调取***，包括：
车辆位置信息获取模块，用于获取车辆定位或车辆的导航路径；
自动驾驶模型调取模块，用于当车辆定位或车辆的导航路径在预设范围时，调取路侧设备中的自动驾驶模型对车辆进行自动驾驶行为指导，当不在所述预设范围内时，调取云端服务器中的自动驾驶模型对车辆进行自动驾驶行为指导，所述自动驾驶模型基于第一方面所述的云路协同的自动驾驶模型训练方法得到。
第五方面，本申请实施例提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机指令，所述计算机指令用于使所述计算机执行本申请实施例第二个方面所述的云路协同的自动驾驶模型调取方法。
第六方面，本申请实施例提供一种终端，包括：存储器和处理器，所述存储器和所述处理器之间互相通信连接，所述存储器存储有计算机指令，所述处理器通过执行所述计算机指令，从而执行本申请实施例第二个方面所述的云路协同的自动驾驶模型调取方法。
本申请技术方案，具有如下优点：
1、本申请实施例提供的云路协同的自动驾驶模型训练方法及***，将能车载传感设备将采集的车辆状态及路侧设备存储的道路数据进行融合，可以针对个人或者路段等因素进行个性化的训练与优化，路侧设备可以通过路侧的车联网协议接收车辆状态数据结合自己的本地数据集进行模型更新，可以节省数据传输的宽带，同时与云端服务器的强力计算进行整合提高了计算效率，其训练过程结合驾驶员的介入信息使车载***能够自适应进行训练，不需要人为的介入即可持续提升自动驾驶模型的识别效果，实用性强，可以提升驾驶体验。
2、本申请实施例提供的云路协同的自动驾驶模型调取方法及***，根据车辆的定位及导航路径结合历史轨迹范围及历史轨迹频率确定的预设范围，选择优先调取更具个性化的路侧设备终端的自动驾驶模型，还是优先调取更具泛化能 力的云端服务器的自动驾驶模型进行自动驾驶指导，可以达到更加精确的自动驾驶识别效果，可以给用户更好的驾驶体验。
附图说明
为了更清楚地说明本申请具体实施方式或现有技术中的技术方案，下面将对具体实施方式或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图是本申请的一些实施方式，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本申请实施例中提供的云路协同的自动驾驶模型训练方法的一个示例的流程图；
图2为本申请实施例中云路协同的自动驾驶模型调取方法的一个示例的流程图；
图3为本申请实施例中提供的云路协同的自动驾驶模型调取***的一个具体示例的模块组成图；
图4为本申请实施例提供的终端一个具体示例的组成图。
具体实施方式
下面将结合附图对本申请的技术方案进行清楚、完整地描述，显然，所描述的实施例是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
此外，下面所描述的本申请不同实施方式中所涉及的技术特征只要彼此之间未构成冲突就可以相互结合。
实施例1
本申请实施例提供一种云路协同的自动驾驶模型训练方法，如图1所示，该方法包括如下步骤：
步骤S11：车载传感设备将采集的车辆状态数据通过车联网协议实时发送至路侧设备。
步骤S12：路侧设备对车载传感设备新上传的数据与之前已经储存的数据合并为数据集，并对该数据集进行预处理。
步骤S13：车载控制器控制将预处理后的数据集在路侧设备进行自动驾驶模型训练更新，或控制将预处理后的数据集传输至云端服务器进行自动驾驶模型训练更新；
步骤S141：当基于预处理后的数据集在路侧设备对自动驾驶模型进行训练更新时，将更新后的模型上传至云端服务器；
步骤S142：当云端服务器进行自动驾驶模型训练更新时，将预处理后的数据集通过以太网传输至云端服务器，与云端服务器中的数据集一起训练自动驾驶模型，将更新后的自动驾驶模型反馈至路侧设备。
本申请实施例提供的云路协同的自动驾驶模型训练方法，将能车载传感设备将采集的车辆状态及路侧设备存储的道路数据进行融合，基于该融合数据可以针对个人或者路段等因素进行个性化的训练与优化，同时机器学习的识别效果很大程度取决于数据的量与多样性，而对于汽车终端来说，制约数据量的条件是网络宽带，本申请实施例将数据通过路侧设备与汽车的车联网协议(例如是v2i网络协议)进行传输，提供了传输效率，然后使用路侧部署的计算节点，对汽车上传的数据与之前路侧存储的数据合并为一个数据集，进行数据的预处理。
本申请实施例经过路侧设备对数据进行预处理后，数据量会下降很多，可以更加快速的将数据传输给云端服务器进行任务调度，然后与云端的数据集一起训练新的模型，再将模型反馈到路侧设备；如果路侧设备拥有的计算节点足够强力，可以承担一部分模型的训练工作，此时路侧设备可以通过路侧的车联网协议接收车辆状态数据结合自己的本地数据集进行进行模型更新，可以节省数据传输的宽带，仅需将任务处理结果的交互信息与云端服务器进行传输即可。
本申请实施例提供的车载传感设备新上传的数据包括：驾驶员对之前已有自动驾驶模型指导下形成行驶行为的介入数据；预处理的过程包括根据驾驶员的介入数据进行真值标注及数据清洗。
实际应用中协同车载传感设备与驾驶员是否介入的这个因素对采集到的数据的真值进行判断，比如其他多个传感器识别到了有障碍物，而图像设备采集的数据没有识别到，对应的就认为图像识别模型对于该物体的识别效果不好，可以结合传感器数据对该图像的原始数据进行标记，将该数据传输给自动驾驶模型进行进一步训练。当有驾驶员介入时，以驾驶员的判断为真值，将一些传感数据的错误信息进行标注，然后将这些错误数据进行处理，就可以提升自动驾驶模型 的识别精度。
在一具体实施例中，对于模型更新的部分，可以使用动态加载技术通过编写通用的接口，使得车载***可以插扩式对模块进行自由加载与卸载，配合生命周期管理，与远程下载平台，可以通过生命周期对***资源进行调配与管理，使得应用程序可以更简单地进行扩展与维护，在汽车终端处于合适的状态时，通过动态加载即可重新加载识别模块与自动驾驶模型。
与上述方法对应的，本申请实施例还提供一种云路协同的自动驾驶模型训练***，包括：车载传感设备、车载控制器及云端服务器，其中车载传感设备，用于将采集的车辆状态数据通过车联网协议实时发送至路侧设备；路侧设备，用于对车载传感设备新上传的数据与之前已经储存的数据合并为数据集，并对该数据集进行预处理；车载控制器，用于控制将预处理后的数据集在路侧设备进行自动驾驶模型训练更新，或控制将预处理后的数据集传输至云端服务器进行自动驾驶模型训练更新；路侧设备还用于基于预处理后的数据集在路侧设备对自动驾驶模型进行训练更新，并将更新后的模型上传至云端服务器；云端服务器，用于获取路侧设备预处理后的数据集并将其已存储的数据集一起训练自动驾驶模型，将更新后的自动驾驶模型反馈至路侧设备。
本申请实施例提供的云路协同的自动驾驶模型训练方法及***，将能车载传感设备将采集的车辆状态及路侧设备存储的道路数据进行融合，可以针对个人或者路段等因素进行个性化的训练与优化，路侧设备可以通过路侧的车联网协议接收车辆状态数据结合自己的本地数据集进行进行模型更新，可以节省数据传输的宽带，同时与云端服务器的强力计算进行整合提高了计算效率，其训练过程结合驾驶员的介入信息使车载***能够自适应进行训练，不需要人为的介入即可持续提升自动驾驶模型的识别效果，实用性强，可以提升驾驶体验。
实施例2
本申请实施例提供一种云路协同的自动驾驶模型调取方法，如图2所示，包括：
步骤S21：获取车辆定位或车辆的导航路径。
步骤S22：当车辆定位或车辆的导航路径在预设范围时，调取路侧设备中的自动驾驶模型对车辆进行自动驾驶行为指导，当不在所述预设范围内时，调取云端服务 器中的自动驾驶模型对车辆进行自动驾驶行为指导。
本申请实施例中，所述预设范围根据车辆的历史轨迹范围及历史轨迹频率确定，例如车辆定位是车主的家地点和工作地点之间必经的路线，或者周末经常活动的路线，则优选调取路侧设备中的自动驾驶模型，如果不是车主经常活动的路线，例如是其他城市的自驾游，则调取路侧设备终端的自动驾驶模型。
由于路侧设备主要针对目标车辆特定轨迹范围的内数据采集和预处理，云端服务器是接收所有路侧设备上传的模型或预处理的数据，因此云端服务器比路侧设备中存储的自动驾驶模型更具泛化性，而路侧设备存储的自动驾驶模型相比云端服务器更具个性化，因此在车辆在实施自动驾驶模式时，需要首先获取车辆定位或车辆的导航路径，是车辆经常活动的范围，如果是车载控制器则优先调取更具个性化的路侧设备终端的自动驾驶模型，如果不是车辆经常活动的范围则优先调取更具泛化能力的云端服务器的自动驾驶模型。
与上述云路协同的自动驾驶模型调取方法相对应的，本申请实施例还提一种云路协同的自动驾驶模型调取***，如图3所示，包括：
车辆位置信息获取模块21，用于获取车辆定位或车辆的导航路径；该模块为集成定位和导航功能的功能模块，此模块执行实施例1中的步骤S21所描述的方法，在此不再赘述。
自动驾驶模型调取模块22，用于当车辆定位或车辆的导航路径在预设范围时，调取路侧设备中的自动驾驶模型对车辆进行自动驾驶行为指导，当不在所述预设范围内时，调取云端服务器中的自动驾驶模型对车辆进行自动驾驶行为指导。此模块执行实施例1中的步骤S22所描述的方法，在此不再赘述。
本申请实施例提供的云路协同的自动驾驶模型调取方法及***，根据车辆的定位及导航路径结合历史轨迹范围及历史轨迹频率确定的预设范围，选择优先调取更具个性化的路侧设备终端的自动驾驶模型，还是优先调取更具泛化能力的云端服务器的自动驾驶模型进行自动驾驶指导，可以达到更加精确的自动驾驶识别效果，可以给用户更好的驾驶体验。
实施例3
本申请实施例提供一种终端，如图4所示，该设备可以包括处理器51和存储器52，其中处理器51和存储器52可以通过总线或者其他方式连接，图4以通过总线连接为例。
存储器52作为一种非暂态计算机可读存储介质，可用于存储非暂态软件程序、非暂态计算机可执行程序以及模块，如本申请实施例中的对应的程序指令/模块。处理器51通过运行存储在存储器52中的非暂态软件程序、指令以及模块，从而执行处理器的各种功能应用以及数据处理，即实现上述方法实施例2中的云路协同的自动驾驶模型调取方法。
存储器52可以包括存储程序区和存储数据区，其中，存储程序区可存储操作***、至少一个功能所需要的应用程序；存储数据区可存储处理器51所创建的数据等。此外，存储器52可以包括高速随机存取存储器，还可以包括非暂态存储器，例如至少一个磁盘存储器件、闪存器件、或其他非暂态固态存储器件。在一些实施例中，存储器52可选包括相对于处理器51远程设置的存储器，这些远程存储器可以通过网络连接至处理器51。上述网络的实例包括但不限于互联网、企业内部网、企业内网、移动通信网及其组合。
一个或者多个模块存储在存储器52中，当被处理器51执行时，执行实施例2中的云路协同的自动驾驶模型调取方法。
上述终端具体细节可以对应参阅实施例2中对应的相关描述和效果进行理解，此处不再赘述。
本领域技术人员可以理解，实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)、随机存储记忆体(Random Access Memory，RAM)、快闪存储器(Flash Memory)、硬盘(Hard Disk Drive，缩写：HDD)或固态硬盘(Solid-State Drive，SSD)等；存储介质还可以包括上述种类的存储器的组合。
显然，上述实施例仅仅是为清楚地说明所作的举例，而并非对实施方式的限定。对于所属领域的普通技术人员来说，在上述说明的基础上还可以做出其它不同形式的变化或变动。这里无需也无法对所有的实施方式予以穷举。而由此所引申出的显而易见的变化或变动仍处于本申请创造的保护范围之中。

Claims (8)

  1. 一种云路协同的自动驾驶模型训练方法，其特征在于，包括如下步骤：
    车载传感设备将采集的车辆状态数据通过车联网协议实时发送至路侧设备；
    路侧设备对车载传感设备新上传的数据与之前已经储存的数据合并为数据集，并对该数据集进行预处理；
    车载控制器控制将预处理后的数据集在路侧设备进行自动驾驶模型训练更新；所述预处理的过程包括根据驾驶员的介入数据进行真值标注及数据清洗；
    当基于预处理后的数据集在路侧设备对自动驾驶模型进行训练更新时，将更新后的模型上传至云端服务器。
  2. 根据权利要求1所述的云路协同的自动驾驶模型训练方法，其特征在于，车载传感设备新上传的数据包括：驾驶员对之前已有自动驾驶模型指导下形成行驶行为的介入数据。
  3. 根据权利要求2所述的云路协同的自动驾驶模型训练方法，其特征在于，预处理的过程包括根据驾驶员的介入数据进行真值标注及数据清洗。
  4. 一种云路协同的自动驾驶模型调取方法，其特征在于，包括以下步骤：
    获取车辆定位或车辆的导航路径；
    当车辆定位或车辆的导航路径在预设范围时，调取路侧设备中的自动驾驶模型对车辆进行自动驾驶行为指导，当不在所述预设范围内时，调取云端服务器中的自动驾驶模型对车辆进行自动驾驶行为指导，所述自动驾驶模型基于权利要求1-3任一所述的云路协同的自动驾驶模型训练方法得到，所述预设范围根据车辆的历史轨迹范围及历史轨迹频率确定。
  5. 一种云路协同的自动驾驶模型训练***，其特征在于，包括：车载传感设备、车载控制器及云端服务器，其中：
    车载传感设备，用于将采集的车辆状态数据通过车联网协议实时发送至路侧设备；
    路侧设备，用于对车载传感设备新上传的数据与之前已经储存的数据合并为数据集，并对该数据集进行预处理；
    车载控制器，用于控制将预处理后的数据集在路侧设备进行自动驾驶模型训练更新；
    路侧设备还用于基于预处理后的数据集在路侧设备对自动驾驶模型进行训练更新，并将更新后的模型上传至云端服务器；
    云端服务器，用于获取路侧设备预处理后的数据集并将其已存储的数据集一起训练自动驾驶模型，将更新后的自动驾驶模型反馈至路侧设备。
  6. 一种云路协同的自动驾驶模型调取***，其特征在于，包括：
    车辆位置信息获取模块，用于获取车辆定位或车辆的导航路径；
    自动驾驶模型调取模块，用于当车辆定位或车辆的导航路径在预设范围时，调取路侧设备中的自动驾驶模型对车辆进行自动驾驶行为指导，当不在所述预设范围内时，调取云端服务器中的自动驾驶模型对车辆进行自动驾驶行为指导，所述自动驾驶模型基于权利要求1-3任一所述的云路协同的自动驾驶模型训练方法得到，所述预设范围根据车辆的历史轨迹范围及历史轨迹频率确定。
  7. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机指令，所述计算机指令用于使所述计算机执行如权利要求4所述的方法。
  8. 一种终端，其特征在于，包括：存储器和处理器，所述存储器和所述处理器之间互相通信连接，所述存储器存储有计算机指令，所述处理器通过执行所述计算机指令，从而执行如权利要求4所述的方法。
PCT/CN2021/144059 2021-01-25 2021-12-31 一种云路协同的自动驾驶模型训练、调取方法及*** WO2022156520A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110093555.5A CN112463347B (zh) 2021-01-25 2021-01-25 一种云路协同的自动驾驶模型训练、调取方法及***
CN202110093555.5 2021-01-25

Publications (1)

Publication Number Publication Date
WO2022156520A1 true WO2022156520A1 (zh) 2022-07-28

Family

ID=74802372

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/144059 WO2022156520A1 (zh) 2021-01-25 2021-12-31 一种云路协同的自动驾驶模型训练、调取方法及***

Country Status (2)

Country Link
CN (1) CN112463347B (zh)
WO (1) WO2022156520A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115473750A (zh) * 2022-08-04 2022-12-13 湖南湘江智车出行科技有限公司 基于云边协同的路侧设备能耗优化控制方法及装置
CN115474176A (zh) * 2022-08-22 2022-12-13 武汉大学 自动驾驶地图中车-路-云三端数据的交互方法及设备
CN115499315A (zh) * 2022-09-14 2022-12-20 北京百度网讯科技有限公司 Ota升级任务的执行方法、装置和电子设备
CN116013092A (zh) * 2022-12-07 2023-04-25 嘉应学院 一种基于云计算的道路交通管理***
CN116153090A (zh) * 2022-10-25 2023-05-23 浙江特锐讯智能科技有限公司 一种全轨迹精准化车牌识别管理方法及***
CN116503029A (zh) * 2023-06-27 2023-07-28 北京中电科卫星导航***有限公司 用于自动驾驶的模块数据协同处理方法及***
CN116595384A (zh) * 2023-07-14 2023-08-15 支付宝(杭州)信息技术有限公司 模型训练方法及装置
CN117727183A (zh) * 2024-02-18 2024-03-19 南京淼瀛科技有限公司 结合车路协同的自动驾驶安全预警方法及***

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112463347B (zh) * 2021-01-25 2021-06-11 国汽智控(北京)科技有限公司 一种云路协同的自动驾驶模型训练、调取方法及***
CN113630449B (zh) * 2021-07-26 2023-12-26 中汽创智科技有限公司 一种车云数据传输***、方法、设备及存储介质
CN113401132B (zh) * 2021-07-29 2022-10-18 国汽智控(北京)科技有限公司 一种驾驶模型更新方法、装置及电子设备
CN114056350A (zh) * 2021-11-08 2022-02-18 国汽智控(北京)科技有限公司 一种用于自动驾驶的控制方法、装置、设备及介质
CN114384906B (zh) * 2021-12-01 2024-02-02 合肥湛达智能科技有限公司 一种路边单元与智能网联汽车计算任务分配方法
CN114338753A (zh) * 2022-03-10 2022-04-12 国汽智控(北京)科技有限公司 一种车云协同计算***、设备及存储介质
CN114742236A (zh) * 2022-04-24 2022-07-12 重庆长安汽车股份有限公司 一种环境车辆行为预测模型训练方法及***

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109377778A (zh) * 2018-11-15 2019-02-22 济南浪潮高新科技投资发展有限公司 一种基于多路rdma和v2x的协同自动驾驶***及方法
CN110083163A (zh) * 2019-05-20 2019-08-02 三亚学院 一种用于自动驾驶汽车的5g c-v2x车路云协同感知方法及***
KR20190099148A (ko) * 2019-08-05 2019-08-26 엘지전자 주식회사 자율 주행 시스템에서 데이터 기반의 어플레이케이션 갱신방법 및 이를 위한 장치
CN110750052A (zh) * 2019-09-30 2020-02-04 奇点汽车研发中心有限公司 驾驶模型的训练方法和装置、电子设备和介质
CN111211893A (zh) * 2020-01-22 2020-05-29 西安电子科技大学 一种应用区块链的自动驾驶汽车模型共享方法
CN112116031A (zh) * 2020-10-29 2020-12-22 重庆长安汽车股份有限公司 基于路侧设备的目标融合方法、***、车辆及存储介质
CN112463347A (zh) * 2021-01-25 2021-03-09 国汽智控(北京)科技有限公司 一种云路协同的自动驾驶模型训练、调取方法及***

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104484911B (zh) * 2014-10-29 2016-08-24 云南大学 基于QoE的个性化自动驾驶参数优化设定方法
US11273836B2 (en) * 2017-12-18 2022-03-15 Plusai, Inc. Method and system for human-like driving lane planning in autonomous driving vehicles
CN108549384A (zh) * 2018-05-21 2018-09-18 济南浪潮高新科技投资发展有限公司 一种5g环境下远程控制自动驾驶方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109377778A (zh) * 2018-11-15 2019-02-22 济南浪潮高新科技投资发展有限公司 一种基于多路rdma和v2x的协同自动驾驶***及方法
CN110083163A (zh) * 2019-05-20 2019-08-02 三亚学院 一种用于自动驾驶汽车的5g c-v2x车路云协同感知方法及***
KR20190099148A (ko) * 2019-08-05 2019-08-26 엘지전자 주식회사 자율 주행 시스템에서 데이터 기반의 어플레이케이션 갱신방법 및 이를 위한 장치
CN110750052A (zh) * 2019-09-30 2020-02-04 奇点汽车研发中心有限公司 驾驶模型的训练方法和装置、电子设备和介质
CN111211893A (zh) * 2020-01-22 2020-05-29 西安电子科技大学 一种应用区块链的自动驾驶汽车模型共享方法
CN112116031A (zh) * 2020-10-29 2020-12-22 重庆长安汽车股份有限公司 基于路侧设备的目标融合方法、***、车辆及存储介质
CN112463347A (zh) * 2021-01-25 2021-03-09 国汽智控(北京)科技有限公司 一种云路协同的自动驾驶模型训练、调取方法及***

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115473750A (zh) * 2022-08-04 2022-12-13 湖南湘江智车出行科技有限公司 基于云边协同的路侧设备能耗优化控制方法及装置
CN115473750B (zh) * 2022-08-04 2024-06-11 湖南湘江智车出行科技有限公司 基于云边协同的路侧设备能耗优化控制方法及装置
CN115474176B (zh) * 2022-08-22 2024-03-08 武汉大学 自动驾驶地图中车-路-云三端数据的交互方法及设备
CN115474176A (zh) * 2022-08-22 2022-12-13 武汉大学 自动驾驶地图中车-路-云三端数据的交互方法及设备
CN115499315A (zh) * 2022-09-14 2022-12-20 北京百度网讯科技有限公司 Ota升级任务的执行方法、装置和电子设备
CN116153090A (zh) * 2022-10-25 2023-05-23 浙江特锐讯智能科技有限公司 一种全轨迹精准化车牌识别管理方法及***
CN116153090B (zh) * 2022-10-25 2024-03-29 浙江特锐讯智能科技有限公司 一种全轨迹精准化车牌识别管理方法及***
CN116013092A (zh) * 2022-12-07 2023-04-25 嘉应学院 一种基于云计算的道路交通管理***
CN116503029B (zh) * 2023-06-27 2023-09-05 北京中电科卫星导航***有限公司 用于自动驾驶的模块数据协同处理方法及***
CN116503029A (zh) * 2023-06-27 2023-07-28 北京中电科卫星导航***有限公司 用于自动驾驶的模块数据协同处理方法及***
CN116595384B (zh) * 2023-07-14 2023-11-24 支付宝(杭州)信息技术有限公司 模型训练方法及装置
CN116595384A (zh) * 2023-07-14 2023-08-15 支付宝(杭州)信息技术有限公司 模型训练方法及装置
CN117727183A (zh) * 2024-02-18 2024-03-19 南京淼瀛科技有限公司 结合车路协同的自动驾驶安全预警方法及***
CN117727183B (zh) * 2024-02-18 2024-05-17 南京淼瀛科技有限公司 结合车路协同的自动驾驶安全预警方法及***

Also Published As

Publication number Publication date
CN112463347B (zh) 2021-06-11
CN112463347A (zh) 2021-03-09

Similar Documents

Publication Publication Date Title
WO2022156520A1 (zh) 一种云路协同的自动驾驶模型训练、调取方法及***
DE102018121595B4 (de) Unbeaufsichtigtes anlernen von agenten für autonome fahranwendungen
CN109131345B (zh) 交通工具和控制交通工具的方法和***
US10334049B2 (en) Highly assisted driving platform
US10259468B2 (en) Active vehicle performance tuning based on driver behavior
DE102019113856A1 (de) Systeme, verfahren und steuerungen für ein autonomes fahrzeug, die autonome fahreragenten und fahrrichtlinienlernende implementieren, um richtlinien basierend auf kollektiven fahrerfahrungen der autonomen fahreragenten zu erstellen und zu verbessern
WO2019047651A1 (zh) 驾驶行为预测方法和装置、无人车
CN108292474A (zh) 派遣和维护自主车辆的车队的协调
JP6567170B2 (ja) 自動化された自動車の位置特定方法
DE102019115038A1 (de) Systeme und verfahren zur vorhersage von sensorinformationen
CN106652515A (zh) 车辆自动控制方法、装置和***
CN113135178A (zh) 泊车路线共享方法、装置、设备以及存储介质
US20230391357A1 (en) Methods and apparatus for natural language based scenario discovery to train a machine learning model for a driving system
US12013251B2 (en) Dynamic map generation with focus on construction and localization field of technology
KR102607390B1 (ko) 차량의 주변 상황 확인 방법
JP6238738B2 (ja) 経路探索装置及びその方法、並びに経路を探索するためのコンピュータプログラム及びコンピュータプログラムを記録した記録媒体
CN114637770A (zh) 车辆轨迹的预测方法、装置
JP6494810B2 (ja) 自動車のデータをアップロードするための方法及び装置
CN107764276A (zh) 一种基于视频监控的导航规划***
CN113422797B (zh) 一种用于车联网的更新地图的方法及***
US20220043459A1 (en) Systems and Methods for Relaying Requests to Autonomous Vehicles
US11301782B2 (en) Method and arrangement for determining a current location speed-limit in a road-vehicle
CN114945033A (zh) 车端数据回传方法、车端控制器、云端服务器及车辆
CN115439957B (zh) 一种智能驾驶数据采集方法、采集装置、采集设备及一种计算机可读存储介质
CN116385825B (zh) 模型联合训练方法、装置及车辆

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21920891

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21920891

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21920891

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 11.01.2024)