CN116244974A - Method and device for evaluating authenticity of simulated traffic flow data - Google Patents

Method and device for evaluating authenticity of simulated traffic flow data Download PDF

Info

Publication number
CN116244974A
CN116244974A CN202310489520.2A CN202310489520A CN116244974A CN 116244974 A CN116244974 A CN 116244974A CN 202310489520 A CN202310489520 A CN 202310489520A CN 116244974 A CN116244974 A CN 116244974A
Authority
CN
China
Prior art keywords
traffic flow
flow data
distribution
driving parameters
scene
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310489520.2A
Other languages
Chinese (zh)
Inventor
郝坤坤
王江成
潘余曦
杨子江
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xi'an Xinxin Information Technology Co ltd
Original Assignee
Xi'an Xinxin Information Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xi'an Xinxin Information Technology Co ltd filed Critical Xi'an Xinxin Information Technology Co ltd
Priority to CN202310489520.2A priority Critical patent/CN116244974A/en
Publication of CN116244974A publication Critical patent/CN116244974A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00Computer-aided design [CAD]
    • G06F30/20Design optimisation, verification or simulation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2111/00Details relating to CAD techniques
    • G06F2111/08Probabilistic or stochastic CAD
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02TCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00Road transport of goods or passengers
    • Y02T10/10Internal combustion engine [ICE] based vehicles
    • Y02T10/40Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Hardware Design (AREA)
  • Evolutionary Computation (AREA)
  • Geometry (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention provides a method and a device for evaluating the authenticity of simulated traffic flow data, relates to the technical field of automatic driving, and can be applied to the scenes of evaluating the simulated traffic flow data, automatic driving simulation test and the like. The specific implementation scheme comprises the following steps: obtaining simulation traffic flow data and real traffic flow data corresponding to a preset functional scene; aiming at the target preset functional scene, determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene. The method and the device can carry out the authenticity evaluation on the simulated traffic flow data based on the functional scene granularity, the manner of the authenticity evaluation is more flexible, and the accuracy of the authenticity evaluation result of the simulated traffic flow data is higher.

Description

Method and device for evaluating authenticity of simulated traffic flow data
Technical Field
The disclosure relates to the technical field of automatic driving, in particular to a method and a device for evaluating the authenticity of simulated traffic flow data.
Background
The simulation test is a key ring for realizing high-order automatic driving floor application, and can solve the problems of long test period, high cost and low efficiency of the road real vehicle. Traffic flow simulation plays an important role in automated driving vehicle testing as part of the simulation system. Before using the simulated traffic flow data to conduct automatic driving test, it is important to quantitatively evaluate the authenticity of the simulated traffic flow data.
At present, the method for evaluating the authenticity of the simulated traffic flow data mainly comprises the following steps: and calculating the differences of road parameter indexes such as traffic flow, traffic density, saturated flow rate and the like between the simulated traffic flow data and the real traffic flow data from a macroscopic angle, and evaluating the authenticity of the simulated traffic flow data.
However, when the authenticity of the simulated traffic flow data is evaluated according to the current mode, the accuracy of the authenticity evaluation result is still to be improved, and the granularity of the authenticity evaluation is rough and inflexible.
Disclosure of Invention
The invention provides a method and a device for evaluating the authenticity of simulated traffic flow data, which can evaluate the authenticity of the simulated traffic flow data based on the granularity of functional scenes, increase the flexibility of the authenticity evaluation and improve the accuracy of the authenticity evaluation result of the simulated traffic flow data.
According to a first aspect of the present disclosure, there is provided a simulated traffic flow data authenticity assessment method, the method comprising: obtaining simulated traffic flow data and real traffic flow data corresponding to a preset functional scene, wherein the categories of the preset functional scene are divided according to at least one of test functions, static road network structures and dynamic interactive vehicle numbers; aiming at the target preset functional scene, determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene.
In some possible implementations, determining the result of evaluating the authenticity of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene includes: extracting simulation driving parameters in simulation traffic flow data corresponding to a target preset functional scene and real driving parameters in corresponding real traffic flow data; modeling the distribution of the simulation driving parameters and the distribution of the real driving parameters respectively through a flow model to obtain a first probability density distribution of the simulation driving parameters and a second probability density distribution of the real driving parameters; and determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the first probability density distribution and the second probability density distribution.
The similarity between the first probability density distribution and the second probability density distribution is used for representing the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene.
In some possible implementations, the encoder of the stream model implements a reversible transformation through an autoregressive neural network and a cubic spline interpolation algorithm.
In some possible implementations, the modeling, by using the flow model, the distribution of the simulated driving parameter and the distribution of the real driving parameter respectively, to obtain a first probability density distribution of the simulated driving parameter and a second probability density distribution of the real driving parameter, including: modeling the distribution of the simulation driving parameters through a flow model to obtain the log likelihood of the simulation driving parameters; modeling the distribution of the real driving parameters through a flow model to obtain the log likelihood of the real driving parameters; according to a preset log likelihood interval set, respectively counting the log likelihood distribution of the simulation driving parameters and the log likelihood distribution of the real driving parameters.
The log-likelihood distribution of the simulation driving parameters is used for representing the first probability density distribution, and the log-likelihood distribution of the real driving parameters is used for representing the second probability density distribution.
In some possible implementations, determining the result of evaluating the authenticity of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the first probability density distribution and the second probability density distribution includes: determining a distribution area intersection ratio of the log-likelihood distribution of the simulation driving parameters and the log-likelihood distribution of the real driving parameters, wherein the distribution area intersection ratio is used for representing the similarity between the first probability density distribution and the second probability density distribution; and determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the cross-merging ratio of the distribution areas.
In some possible implementations, the authenticity assessment result includes a degree of authenticity; the method further comprises the steps of: obtaining the reality of simulated traffic flow data corresponding to at least two target preset functional scenes; and carrying out weighted summation on the authenticity of the simulated traffic flow data corresponding to the at least two target preset functional scenes to obtain the comprehensive authenticity of the simulated traffic flow data corresponding to the at least two target preset functional scenes.
Optionally, when the reality of the simulated traffic flow data corresponding to at least two target preset functional scenes is weighted and summed, the weight of each target preset functional scene is a preset value.
Illustratively, the simulated driving parameters and the actual driving parameters include: absolute speed of the host vehicle, absolute speed of the dynamically interactive vehicle, relative distance of the host vehicle from the dynamically interactive vehicle, and relative speed.
Illustratively, the preset functional scenario includes at least one of: front vehicle braking scene, front vehicle cut-in scene, main vehicle lane changing scene, main vehicle overtaking scene and crossroad unprotected left turning scene.
The first aspect of the present disclosure has at least the following advantages: the method can perform the authenticity evaluation of the simulated traffic flow data aiming at specific one or more preset functional scenes, realize the authenticity evaluation of the simulated traffic flow data based on the granularity of the functional scenes, and increase the flexibility of the authenticity evaluation, such as: different functional scenes can be flexibly configured to evaluate the authenticity of the simulated traffic flow data, and the accuracy of the authenticity evaluation result of the simulated traffic flow data is greatly improved.
According to a second aspect of the present disclosure, there is provided an emulated traffic flow data authenticity assessment device, the device comprising: a scene dividing unit and an authenticity evaluating unit.
The scene dividing unit is used for acquiring simulated traffic flow data and real traffic flow data corresponding to a preset functional scene, and the categories of the preset functional scene are divided according to at least one of test functions, static road network structures and dynamic interaction vehicle numbers.
The authenticity evaluation unit is used for determining an authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the simulated traffic flow data corresponding to the target preset functional scene and the real traffic flow data.
In some possible implementations, the authenticity assessment unit is specifically configured to: extracting simulation driving parameters in simulation traffic flow data corresponding to a target preset functional scene and real driving parameters in corresponding real traffic flow data; modeling the distribution of the simulation driving parameters and the distribution of the real driving parameters respectively through a flow model to obtain a first probability density distribution of the simulation driving parameters and a second probability density distribution of the real driving parameters; and determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the first probability density distribution and the second probability density distribution.
The similarity between the first probability density distribution and the second probability density distribution is used for representing the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene.
In some possible implementations, the encoder of the stream model implements a reversible transformation through an autoregressive neural network and a cubic spline interpolation algorithm.
In some possible implementations, the authenticity assessment unit is specifically configured to: modeling the distribution of the simulation driving parameters through a flow model to obtain the log likelihood of the simulation driving parameters; modeling the distribution of the real driving parameters through a flow model to obtain the log likelihood of the real driving parameters; according to a preset log likelihood interval set, respectively counting the log likelihood distribution of the simulation driving parameters and the log likelihood distribution of the real driving parameters.
The log-likelihood distribution of the simulation driving parameters is used for representing the first probability density distribution, and the log-likelihood distribution of the real driving parameters is used for representing the second probability density distribution.
In some possible implementations, the authenticity assessment unit is specifically configured to: determining a distribution area intersection ratio of the log-likelihood distribution of the simulation driving parameters and the log-likelihood distribution of the real driving parameters, wherein the distribution area intersection ratio is used for representing the similarity between the first probability density distribution and the second probability density distribution; and determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the cross-merging ratio of the distribution areas.
In some possible implementations, the authenticity assessment result includes a degree of authenticity; the authenticity evaluation unit is further configured to: obtaining the reality of simulated traffic flow data corresponding to at least two target preset functional scenes; and carrying out weighted summation on the authenticity of the simulated traffic flow data corresponding to the at least two target preset functional scenes to obtain the comprehensive authenticity of the simulated traffic flow data corresponding to the at least two target preset functional scenes.
Optionally, when the reality of the simulated traffic flow data corresponding to at least two target preset functional scenes is weighted and summed, the weight of each target preset functional scene is a preset value.
Illustratively, the simulated driving parameters and the actual driving parameters include: absolute speed of the host vehicle, absolute speed of the dynamically interactive vehicle, relative distance of the host vehicle from the dynamically interactive vehicle, and relative speed.
Illustratively, the preset functional scenario includes at least one of: front vehicle braking scene, front vehicle cut-in scene, main vehicle lane changing scene, main vehicle overtaking scene and crossroad unprotected left turning scene.
According to a third aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of the first aspect.
According to a fourth aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method according to the first aspect.
According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method according to the first aspect.
The advantages of the second to fifth aspects of the present disclosure may refer to those of the first aspect, and will not be described in detail.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
fig. 1 is a flow chart of a method for evaluating the authenticity of simulated traffic flow data according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of a front vehicle brake test scenario provided in an embodiment of the present disclosure;
fig. 3 is a schematic diagram of a lane change (2 neighboring vehicles) test scenario provided by an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of an overtaking (4 adjacent vehicle) test scenario provided by an embodiment of the present disclosure;
FIG. 5 is a schematic diagram of an AV left turn (BV on the opposite side) test scenario provided by an embodiment of the present disclosure;
FIG. 6 is a schematic flow chart of an implementation of S102 in FIG. 1 according to an embodiment of the disclosure;
FIG. 7 is a schematic flow chart of an implementation of S602 in FIG. 6 according to an embodiment of the disclosure;
fig. 8 is a schematic flow chart of an implementation of S603 in fig. 6 provided in an embodiment of the present disclosure;
fig. 9 is another flow chart of a method for evaluating the authenticity of simulated traffic flow data according to an embodiment of the present disclosure;
FIG. 10 is a schematic flow chart of another exemplary method for evaluating the authenticity of simulated traffic flow data according to an embodiment of the present disclosure;
fig. 11 is a schematic diagram of a composition of a simulated traffic flow data authenticity assessment device provided by an embodiment of the present disclosure;
fig. 12 is a schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure.
Description of the embodiments
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be appreciated that in embodiments of the present disclosure, the character "/" generally indicates that the context associated object is an "or" relationship. The terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated.
In order to facilitate a clearer understanding of the embodiments shown in the present disclosure, before describing the embodiments of the present disclosure, a part of the concepts or words involved in the present disclosure will be first described.
1. Simulation traffic flow: the method is to establish traffic flow conforming to natural driving distribution according to the driving state or rule in the real traffic flow data set in a simulation environment.
2. Specific functional test scenario: in the present disclosure, the test scenario is designed for typical functions of an automatic driving automobile, such as following, lane changing, overtaking, and the like.
3. Flow model: is a generation model that can model the distribution of discrete data samples and infinitely generate data approximating the distribution of real samples by a sampling method.
4. And (3) spline: is a way of interpolating data in which a single formula given by a polynomial satisfies all data points. Whereas splines use multiple formulas, each of which is a lower order polynomial, to pass all data points, cubic splines represent the lower order polynomial used as a third order polynomial.
5. Autoregressive: one way to build a sequential data model is to use the time series of one variable as the dependent variable series, use the time series of the same variable over several periods as the independent variable series, and analyze the correlation between one dependent variable series and another or more independent variable series.
The following begins with an introduction to the embodiments of the present disclosure.
With advances in awareness and decision making technology, autopilot systems have evolved rapidly over the past few years. The precondition that the autopilot system can be deployed on a large scale is that the autopilot system can pass safety tests and verifications. The simulation test is a key ring for realizing high-order automatic driving floor application, and can solve the problems of long test period, high cost and low efficiency of the road real vehicle.
Traffic flow simulation plays an important role in automated driving vehicle testing as part of the simulation system. Before using the simulated traffic flow data to conduct automatic driving test, it is important to quantitatively evaluate the authenticity of the simulated traffic flow data.
At present, the method for evaluating the authenticity of the simulated traffic flow data mainly comprises the following steps: and calculating the differences of road parameter indexes such as traffic flow, traffic density, saturated flow rate and the like between the simulated traffic flow data and the real traffic flow data from a macroscopic angle, and evaluating the authenticity of the simulated traffic flow data.
However, when the authenticity of the simulated traffic flow data is evaluated according to the current mode, the accuracy of the authenticity evaluation result is still to be improved, and the granularity of the authenticity evaluation is rough and inflexible.
Under the background technology, the present disclosure provides a method for evaluating the authenticity of the simulated traffic flow data, which can evaluate the authenticity of the simulated traffic flow data according to specific one or more functional scenes, increase the flexibility of evaluating the authenticity of the simulated traffic flow data, and improve the accuracy of the authenticity evaluation result of the simulated traffic flow data.
The simulation traffic flow data authenticity evaluation method provided by the disclosure can be applied to the scenes of evaluating the simulation traffic flow data, automatic driving simulation test and the like. The method may be executed by a computer or a server, or may be executed by other devices having data processing capabilities. The subject of execution of the method is not limited herein. For example, the simulated traffic flow data may be a data server deploying an autopilot simulation system.
In some embodiments, the server may be a single server, or may be a server cluster formed by a plurality of servers. In some implementations, the server cluster may also be a distributed cluster. The present disclosure is not limited to a specific implementation of the server.
The method for evaluating the authenticity of the simulated traffic flow data is exemplarily described below.
Fig. 1 is a flow chart of a method for evaluating the authenticity of simulated traffic flow data according to an embodiment of the present disclosure. As shown in fig. 1, the method may include S101-S102.
S101, obtaining simulation traffic flow data and real traffic flow data corresponding to a preset functional scene.
The categories of the preset functional scenes are divided according to at least one of test functions, static road network structures and dynamic interactive vehicle numbers.
Illustratively, the test functions may include: braking function, lane changing function, overtaking function, turning function, etc. The static road network structure may include: urban road network structure, highway network structure, etc. The dynamic interactive vehicle refers to an approaching vehicle near a tested vehicle (called a host vehicle) in the form process, and specifically can comprise a front vehicle which is positioned on the same lane as the host vehicle, other vehicles positioned on the approaching lane of the host vehicle, and the like.
In some possible examples, the preset function scenario may include, but is not limited to, at least one of: front vehicle braking scene, front vehicle cut-in scene, main vehicle lane changing scene, main vehicle overtaking scene and crossroad unprotected left turning scene. The embodiment of the disclosure does not limit the specific type of the preset functional scene.
For example, for a simulated traffic flow data set and a real traffic flow data set in the field of simulated driving test, the simulated traffic flow data set and the real traffic flow data set may be respectively divided according to preset functional scenes, so as to obtain simulated traffic flow data and real traffic flow data corresponding to each preset functional scene. The preset function scenario described in S101 may include one or more, which is not limited herein.
The simulated traffic flow data set may be a set of simulated traffic flow data used in an autopilot simulation system to simulate an autopilot environment. The real traffic flow data set may be a set of road acquisition data of an autonomous vehicle or a non-autonomous vehicle. The category of the preset functional scene may be preconfigured.
In one example, the preset function scenario may include a front vehicle brake test scenario. For example, fig. 2 is a schematic diagram of a front vehicle brake test scenario provided in an embodiment of the present disclosure. As shown in fig. 2, the road network structure is a one-way two-lane road, and there are two vehicles of the host vehicle and the brake vehicle on the right side (right side in the traveling direction of the host vehicle, similarly, left side is left side in the traveling direction of the host vehicle, and the same below) lane. The running speed (absolute speed) of the host vehicle is
Figure SMS_1
The running speed (absolute speed) of the brake car is +.>
Figure SMS_2
The distance (relative distance) between the main vehicle and the brake vehicle is +.>
Figure SMS_3
. In FIG. 2In the functional scenario shown, the performance of the host vehicle in the event of a preceding vehicle emergency braking can be tested by setting a wide variety of initialization parameters (e.g., speed, relative distance) for both vehicles.
In yet another example, the preset function scenario may further include a lane change (2 neighboring vehicle) test scenario. For example, fig. 3 is a schematic diagram of a lane change (2-neighboring vehicle) test scenario provided by an embodiment of the present disclosure. As shown in fig. 3, the road network structure is a unidirectional two-lane road network, one vehicle exists in the left lane, two vehicles exist in the right lane, and the two vehicles are a main vehicle and a front vehicle respectively. The running speed (absolute speed) of the host vehicle is
Figure SMS_4
The running speed (absolute speed) of the vehicle in front of the host vehicle is +.>
Figure SMS_5
The distance (relative distance) between the host vehicle and the preceding vehicle is +.>
Figure SMS_6
. The running speed (absolute speed) of the left-hand lane vehicle is +.>
Figure SMS_7
The distance (relative distance) between the host vehicle and the left-hand lane vehicle is +.>
Figure SMS_8
. In the functional scenario shown in fig. 3, the lane-changing performance of the host vehicle under the normal driving or braking condition of the preceding vehicle can be tested by setting various initialization parameters (such as speed and relative distance) for three vehicles.
In yet another example, the preset function scenario may also include a cut-in scenario (4-neighboring vehicle) test scenario. For example, fig. 4 is a schematic diagram of an overtaking (4 adjacent vehicle) test scenario provided by an embodiment of the present disclosure. As shown in fig. 4, the road network structure is a unidirectional three-lane, two vehicles exist in the left lane, two vehicles exist in the middle lane, a main vehicle and a front vehicle respectively, and one vehicle exists in the right lane. Driving speed of main vehicleAbsolute speed) is
Figure SMS_10
. The running speed (absolute speed) of the vehicle in front of the left lane is + ->
Figure SMS_13
. The running speed (absolute speed) of the vehicle behind the left lane is +.>
Figure SMS_16
The distance (relative distance) between the left-side lane-rear vehicle and the left-side lane-front vehicle is +.>
Figure SMS_11
The distance (relative distance) between the host vehicle and the vehicle behind the left lane is +.>
Figure SMS_14
. The running speed (absolute speed) of the vehicle in front of the lane in which the host vehicle is located is +.>
Figure SMS_15
The distance (relative distance) between the host vehicle and the vehicle in front of the lane in which the host vehicle is located is +.>
Figure SMS_17
. The running speed (absolute speed) of the right-hand lane vehicle is +.>
Figure SMS_9
The distance (relative distance) between the host vehicle and the right-hand lane vehicle is +.>
Figure SMS_12
. In the functional scenario shown in fig. 4, the overtaking performance of a host vehicle under a front-vehicle blocking or braking condition may be tested by setting a wide variety of initialization parameters (e.g., speed, relative distance) for five vehicles.
In yet another example, the preset function scenario may also include an AV left turn (BV on the opposite) test scenario. For example, FIG. 5 shows an AV left turn (BV on the opposite side) provided by an embodiment of the present disclosureSchematic diagram of test scenario. As shown in fig. 5, the road network structure is a bidirectional four-lane road network. The running speed (absolute speed) of the main vehicle (AV) is
Figure SMS_18
The distance between the main vehicle and the stop line of the crossroad is
Figure SMS_19
. The driving speed (absolute speed) of the opposite-direction driving vehicle (BV) is +.>
Figure SMS_20
The distance between the vehicles which are driven in opposite directions and the stop line of the crossroad is +.>
Figure SMS_21
. In the functional scenario shown in fig. 5, the performance of a host vehicle left turn through an intersection can be tested by setting a wide variety of initialization parameters (e.g., speed, relative distance) for both vehicles.
It should be appreciated that the above examples of the preset functional scenario are all illustrative, and based on the technical solutions of the present disclosure, those skilled in the art may divide the categories of the preset functional scenario according to at least one of a test function, a static road network structure, and a number of dynamic interactive vehicles.
S102, aiming at a target preset functional scene, determining an authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene.
The target preset function scene may be any one or more preset function scenes.
For example, for each preset functional scene, the authenticity evaluation result of the simulated traffic flow data corresponding to the preset functional scene can be determined according to the similarity between the simulated traffic flow data corresponding to the preset functional scene and the real traffic flow data.
Taking N types of preset functional scenes as examples, N is a positive integer, and in S101, simulated traffic flow data and real traffic flow data corresponding to the N types of preset functional scenes respectively may be obtained.
In S102, for each of the N preset functional scenes, for example, a target preset functional scene, the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene may be determined according to the similarity between the simulated traffic flow data corresponding to the target preset functional scene and the real traffic flow data.
In some implementations, the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene may be calculated by a mature similarity measurement algorithm, such as a cosine distance, an euclidean distance, and the like, which is not limited in this disclosure.
The higher the similarity between the simulated traffic flow data corresponding to the target preset functional scene and the real traffic flow data, the closer the simulated traffic flow data corresponding to the target preset functional scene is to the real traffic flow data, namely the more real the simulated traffic flow data corresponding to the target preset functional scene is.
For example, the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene can be converted into a percentage, and the percentage is used as the authenticity for measuring the authenticity of the simulated traffic flow data. That is, the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene may include an authenticity measure for measuring the authenticity of the simulated traffic flow data.
According to the method and the device, the simulated traffic flow data and the real traffic flow data corresponding to the preset functional scene are obtained, then the real evaluation result of the simulated traffic flow data corresponding to the target preset functional scene is determined according to the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene, the real evaluation of the simulated traffic flow data can be realized for one or more specific (preset) functional scenes, the real evaluation of the simulated traffic flow data based on the granularity of the functional scenes is realized, and the flexibility of the real evaluation is increased, such as: different functional scenes can be flexibly configured to evaluate the authenticity of the simulated traffic flow data, and the manner of evaluating the authenticity is more flexible. The authenticity evaluation is carried out on the simulated traffic flow data based on the functional scene granularity, and the accuracy of the authenticity evaluation result of the simulated traffic flow data can be greatly improved.
Fig. 6 is a schematic flowchart of an implementation of S102 in fig. 1 according to an embodiment of the disclosure. As shown in fig. 6, in some possible implementations, the step of determining the result of evaluating the authenticity of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the simulated traffic flow data and the actual traffic flow data corresponding to the target preset functional scene in S102 may include S601-S603.
S601, extracting simulation driving parameters in simulation traffic flow data corresponding to a target preset functional scene and real driving parameters in corresponding real traffic flow data.
It can be appreciated that the driving parameters extracted from the simulated traffic flow data are referred to as simulated driving parameters, and the driving parameters extracted from the real traffic flow data are referred to as real driving parameters.
Illustratively, the simulated driving parameters and the actual driving parameters may include, but are not limited to: absolute speed of the host vehicle, absolute speed of the dynamic interactive vehicle, relative distance and relative speed of the host vehicle and the dynamic interactive vehicle, and the like. The present disclosure is not limited to the specific types of simulated driving parameters and actual driving parameters.
Taking a front vehicle braking scene as an example, the steps of dividing the simulated traffic flow data corresponding to the front vehicle braking scene and extracting the simulated driving parameters can be as follows.
1) First, an acceleration threshold T is determined, for example T may be-2
Figure SMS_22
(meters per square second), then all vehicles in the simulated traffic flow data are searched, whether the time when the acceleration is smaller than T exists or not is judged, and if the time exists, the vehicle information at the time is stored as braking vehicle information, wherein the braking vehicle information comprises information such as vehicle ID, frame ID, position, acceleration and the like.
2) According to stored brake vehicle time information, b frames and f frames are continuously searched for backward and forward along the time, respectively, while ensuring that the acceleration of the searched frames is less than 0 and that the lane ID remains unchanged. The complete vehicle braking scene information of the (b+f) frame can be obtained, and the complete vehicle braking scene information specifically comprises information such as the ID of a braking vehicle, the frame ID, the position and the acceleration.
3) Searching corresponding main vehicle information according to the starting time information of the braking vehicle, and simultaneously ensuring that the frame ID and the time stamp of the main vehicle are consistent with those of the braking vehicle and the front vehicle of the main vehicle is the braking vehicle; and storing the searched parameter data such as absolute speed, relative distance, relative speed and the like of the main vehicle and the braking vehicle.
The steps of dividing the real traffic flow data corresponding to the front vehicle braking scene and extracting the real driving parameters are similar, and are not repeated.
S602, modeling the distribution of the simulation driving parameters and the distribution of the real driving parameters through a flow model respectively to obtain a first probability density distribution of the simulation driving parameters and a second probability density distribution of the real driving parameters.
The simulation driving parameters and the real driving parameters in the corresponding real traffic flow data can be respectively modeled through the flow model after the simulation driving parameters and the real driving parameters in the simulation traffic flow data corresponding to the functional scene are extracted.
Wherein the nature of the stream model is through an encoder
Figure SMS_23
Will input +.>
Figure SMS_24
Coding as hidden variable +.>
Figure SMS_25
And cause
Figure SMS_26
Obeys a standard normal distribution. The flow model can be specifically referred to the following formula (1).
Figure SMS_27
Equation (1).
In the formula (1),
Figure SMS_28
and->
Figure SMS_29
Gaussian distribution is adopted; />
Figure SMS_30
Designed to be reversible, the density change of the sample can be captured by the change of the variable. />
Figure SMS_31
Representation matrix->
Figure SMS_32
A determinant.
In the disclosed embodiments, inputs
Figure SMS_33
May be a simulated driving parameter or a real driving parameter. And modeling the distribution of the simulation driving parameters and the distribution of the real driving parameters through the flow model respectively to obtain the probability density distribution of the simulation driving parameters and the probability density distribution of the real driving parameters. Wherein the probability density distribution of the simulated driving parameters may be referred to as a first probability density distribution and the probability density distribution of the actual driving parameters may be referred to as a second probability density distribution.
S603, determining an authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the first probability density distribution and the second probability density distribution.
The similarity between the first probability density distribution and the second probability density distribution is used for representing the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene.
For example, in this embodiment, the similarity between the first probability density distribution and the second probability density distribution may be calculated to represent the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene. That is, the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene can be determined according to the similarity between the first probability density distribution and the second probability density distribution.
In some implementations, the similarity between the first probability density distribution and the second probability density distribution may also be calculated by a mature similarity measurement algorithm, such as cosine distance, euclidean distance, etc., which is not limited by the present disclosure.
The similarity between the first probability density distribution and the second probability density distribution can be converted into a percentage, and the percentage can be used as the authenticity for measuring the authenticity of the simulated traffic flow data, so that the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene can be obtained.
According to the method, the device and the system, initial parameters such as simulated driving parameters in simulated traffic flow data corresponding to a target preset functional scene and real driving parameters in corresponding real traffic flow data are extracted, after the distribution of the simulated driving parameters and the distribution of the real driving parameters are respectively modeled through a flow model, the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene is determined according to the similarity between the first probability density distribution of the simulated driving parameters and the second probability density distribution of the real driving parameters, the rule of the simulated traffic flow data can be followed, the difference of the simulated traffic flow data and the real traffic flow data can be measured according to the distribution of the simulated traffic flow data and the real driving parameters corresponding to the functional scene, the authenticity of the simulated traffic flow data is quantized, the authenticity of the simulated traffic flow data can be better evaluated, and the more accurate authenticity evaluation result can be obtained.
In some possible implementations, the encoder of the stream model implements a reversible transformation through an autoregressive neural network and a cubic spline interpolation algorithm.
Illustratively, the encoder of the stream model, i.e., in equation (1) above
Figure SMS_34
. The interval [0,1] can be calculated by parameterizing the cubic spline in this embodiment]Mapping to [0,1]]To increase->
Figure SMS_35
Non-linearities of the transformation.
For example, suppose that the interval is divided into K intervals, that is, K cubic polynomials are corresponding, and K+1 nodes are known
Figure SMS_36
. A continuous monotonically differentiable cubic spline with a given boundary derivative across all nodes is then constructed by solving. Assume that a cubic spline S (x) passing through all nodes is a set of cubic polynomials, S (x) is as follows.
Figure SMS_37
In section +.>
Figure SMS_38
Applying;
Figure SMS_39
in section +.>
Figure SMS_40
Applying; …;
Figure SMS_41
in section +.>
Figure SMS_42
And (3) upper part.
To ensure that the spline has a unique solution, monotonically increasing and mapping [0,1] to [0,1], the following equations (2) through (5) are also required.
Figure SMS_43
Equation (2).
Figure SMS_44
Equation (3).
Figure SMS_45
Equation (4).
Figure SMS_46
At most one solution to guarantee the reversible formula (5).
The autoregressive monotonically organized three transformations are implemented as follows.
1) Autoregressive neural network AR
Figure SMS_47
As input, an unconstrained parameter vector with a length of 2K is output +.>
Figure SMS_48
Where i=d, …, D (D is the dimension of the input).
2) Vector quantity
Figure SMS_49
Is split into->
Figure SMS_50
The length is K. Then will->
Figure SMS_51
,/>
Figure SMS_52
Normalized by Softmax, the length and width of K intervals are output.
3) Calculation of K+1 nodes by width and height of K intervals
Figure SMS_53
4) Finally, K+1 node coordinates are brought into a set of cubic polynomials, and S (x) is solved by taking the formulas (2) to (5) as additional conditions.
Taking overtaking scene (4 adjacent vehicles) asFor example, autoregressive cubic spline stream input
Figure SMS_54
= [ absolute speed of host vehicle, relative position of NPCs and host vehicle, relative speed of NPCs and host vehicle]NPCs refers to dynamic interactive vehicles of a host vehicle, and outputs nine-element random vectors obeying normal distribution>
Figure SMS_55
Optionally, when the parameter modeling is performed on the overtaking scene (4 adjacent vehicles) based on the autoregressive cubic spline flow, an absolute speed (ego _speed) of the host vehicle can be taken as an X axis, a relative position (range) representing the host vehicle and the NPC is taken as a Y axis, a relative speed (range_rate) representing the host vehicle and the NPC is taken as a Z axis, and a simulated driving parameter distribution schematic diagram and a real driving parameter distribution schematic diagram are given.
In the embodiment, the encoder of the flow model realizes reversible transformation through the autoregressive neural network and the cubic spline interpolation algorithm, so that simulation driving parameter distribution and real driving parameter distribution can be respectively modeled based on autoregressive cubic spline flow, and the modeled probability density distribution can be better approximated to real sample distribution. The implementation of reversible transformation through the autoregressive neural network and the cubic spline interpolation algorithm can enable the jacobian in the formula (1) to be easier to calculate, and the efficiency of carrying out the authenticity evaluation on the simulated traffic flow data is improved.
Fig. 7 is a schematic flowchart of an implementation of S602 in fig. 6 according to an embodiment of the disclosure. As shown in fig. 7, in some possible implementations, S602 may include S701-S703.
S701, modeling the distribution of the simulation driving parameters through a flow model to obtain the log likelihood of the simulation driving parameters.
S702, modeling the distribution of the real driving parameters through a flow model to obtain the log likelihood of the real driving parameters.
S703, respectively counting the log-likelihood distribution of the simulation driving parameters and the log-likelihood distribution of the real driving parameters according to a preset log-likelihood interval set.
The log-likelihood distribution of the simulation driving parameters is used for representing the first probability density distribution, and the log-likelihood distribution of the real driving parameters is used for representing the second probability density distribution.
Illustratively, in the present embodiment, after the distribution of the simulated driving parameters is modeled by the flow model, the log likelihood of the simulated driving parameters may be calculated. After modeling the distribution of the real driving parameters through the flow model, the log likelihood of the real driving parameters can be calculated. Then, according to a preset log likelihood interval set, the log likelihood distribution of the simulation driving parameters and the log likelihood distribution of the real driving parameters can be respectively counted. The first probability density distribution may be represented by a log-likelihood distribution of the statistically derived simulated driving parameters, and the second probability density distribution may be represented by a log-likelihood distribution of the statistically derived actual driving parameters.
In other words, in this embodiment, the similarity between the log-likelihood distribution of the simulated driving parameter and the log-likelihood distribution of the real driving parameter may be used to characterize the similarity between the first probability density distribution and the second probability density distribution, and the real evaluation result of the simulated traffic flow data corresponding to the target preset functional scene may be determined according to the similarity between the log-likelihood distribution of the simulated driving parameter and the log-likelihood distribution of the real driving parameter.
Illustratively, a set of preset log-likelihood intervals may be acquired prior to performing S703. For example, the log-likelihood interval may be equally divided into L intervals, the L intervals constituting a log-likelihood interval set, L being an integer greater than 1.
Taking the example that the log likelihood interval set includes L intervals, in this embodiment, the simulated driving parameter may be used as an input of the autoregressive cubic spline stream, the log likelihood of the simulated driving parameter is output, and then the log likelihood distribution data of the simulated driving parameter is counted. The log-likelihood distribution data of the simulated driving parameters can be expressed by the following formula (6).
Figure SMS_56
Equation (6).
In the formula (6) of the present invention,
Figure SMS_57
log likelihood distribution representing simulated driving parameters +. >
Figure SMS_58
Representative interval->
Figure SMS_59
Log-likelihood sample total number of internal simulation driving parameters, +.>
Figure SMS_60
The total number of log likelihood samples representing the simulated driving parameters in all intervals.
In this embodiment, the real driving parameter may be used as an input of an autoregressive cubic spline stream, the log likelihood of the real driving parameter may be output, and then the log likelihood distribution data of the real driving parameter may be counted. The log-likelihood distribution data of the real driving parameters can be expressed by the following formula (7).
Figure SMS_61
Equation (7).
In the formula (7) of the present invention,
Figure SMS_62
log-likelihood distribution representing real driving parameters +.>
Figure SMS_63
Representative interval->
Figure SMS_64
Log-likelihood sample total number of internal real driving parameters, +.>
Figure SMS_65
The total number of log likelihood samples representing the actual driving parameters in all intervals.
In the embodiment, the similarity between the first probability density distribution of the simulated driving parameters and the second probability density distribution of the real driving parameters is represented by using the similarity between the log-likelihood distribution of the simulated driving parameters and the log-likelihood distribution of the real driving parameters, so that the probability density distribution of the simulated driving parameters and the probability density distribution of the real driving parameters can be quantized by using the log-likelihood distribution, the efficiency of carrying out the authenticity evaluation on the simulated traffic flow data can be further improved, and the accuracy of the authenticity evaluation result of the simulated traffic flow data is improved.
Fig. 8 is a schematic flowchart of an implementation of S603 in fig. 6 according to an embodiment of the present disclosure. As shown in fig. 8, in some possible implementations, S603 may include S801-S802.
S801, determining the cross ratio of the distribution area of the log-likelihood distribution of the simulation driving parameters and the distribution area of the log-likelihood distribution of the real driving parameters.
Wherein the cross-over ratio of the distribution areas is used to characterize the similarity between the first probability density distribution and the second probability density distribution.
That is, the similarity between the first probability density distribution and the second probability density distribution (or the similarity between the log-likelihood distribution of the simulated driving parameter and the log-likelihood distribution of the real driving parameter) may be quantified by the cross-over ratio of the distribution areas of the log-likelihood distribution of the simulated driving parameter and the log-likelihood distribution of the real driving parameter.
Illustratively, the cross-ratio of the distribution area of the log-likelihood distribution of the simulated driving parameter to the distribution area of the log-likelihood distribution of the real driving parameter can be calculated by the following formula (8).
Figure SMS_66
Equation (8). />
In the formula (8), the expression "a",
Figure SMS_67
representing the cross-over ratio of the distribution areas; />
Figure SMS_68
Representation interval->
Figure SMS_69
The intersection area of the log-likelihood distribution of the internal simulation driving parameter and the log-likelihood distribution of the real driving parameter,
Figure SMS_70
Representation interval->
Figure SMS_71
The sum area of the log-likelihood distribution of the internal simulation driving parameters and the log-likelihood distribution of the real driving parameters.
S802, determining an authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the cross-merging ratio of the distribution areas.
For example, the cross-plot ratio of the distribution areas obtained in S801 may be used as a measure for the authenticity of the simulated traffic flow data corresponding to the target preset functional scene, to obtain an authenticity evaluation result.
According to the embodiment, the similarity between the log-likelihood distribution of the simulation driving parameters and the log-likelihood distribution of the real driving parameters is quantified by using the cross-correlation ratio of the log-likelihood distribution of the simulation driving parameters and the distribution area of the log-likelihood distribution of the real driving parameters, so that the similarity between the simulation traffic flow data and the real traffic flow data is quantified, the real evaluation result of the simulation traffic flow data corresponding to the target preset functional scene is obtained, the real evaluation efficiency of the simulation traffic flow data can be further improved, and the accuracy of the real evaluation result of the simulation traffic flow data is improved.
Optionally, in some embodiments, a distribution statistical chart of the log-likelihood distribution of the simulated driving parameters and the log-likelihood distribution of the real driving parameters may be further provided, so that the result is presented to the testers more intuitively, and the testers can analyze the simulated traffic flow data more accurately.
As described in the foregoing embodiments, the authenticity evaluation result may include the authenticity. Fig. 9 is another flow chart of a method for evaluating the authenticity of simulated traffic flow data according to an embodiment of the present disclosure. As shown in fig. 9, in some possible implementations, the method may further include S901-S902.
And S901, obtaining the reality of the simulated traffic flow data corresponding to each preset functional scene.
For example, the reality of the simulated traffic flow data corresponding to each preset functional scene may be referred to the foregoing embodiments, and will not be described herein. For example, the reality of the simulated traffic flow data corresponding to a certain preset functional scene may be the intersection ratio of the distribution area of the log-likelihood distribution of the simulated driving parameter corresponding to the preset functional scene and the distribution area of the log-likelihood distribution of the real driving parameter.
And S902, carrying out weighted summation on the reality of the simulated traffic flow data corresponding to all the preset functional scenes to obtain the comprehensive reality of the simulated traffic flow data set.
As described in the foregoing embodiment, the realism of the simulated traffic flow data corresponding to each of the preset functional scenes may be represented by the cross-ratio of the distribution areas of the log-likelihood distribution of the simulated driving parameters corresponding to each of the preset functional scenes and the log-likelihood distribution of the real driving parameters.
In S902, the intersection ratio of the log-likelihood distribution of the simulated driving parameters corresponding to each preset functional scene and the distribution area of the log-likelihood distribution of the real driving parameters may be weighted and summed to obtain the comprehensive reality of the simulated traffic flow data set.
For example, the integrated reality of the simulated traffic flow data set can be calculated by the following formula (9).
Figure SMS_72
Equation (9).
In the formula (9) of the present invention,
Figure SMS_73
representing the comprehensive fidelity of the simulated traffic flow dataset; />
Figure SMS_74
Representing all functionsA set of scenes; />
Figure SMS_75
Representing a single one of the functional scenarios; />
Figure SMS_76
Representing summation of the cross-merging ratios (i.e. the realism) of the distribution areas corresponding to all the preset functional scenes; />
Figure SMS_77
Representation set->
Figure SMS_78
The number of categories of the preset functional scene is included in the system, such as N categories.
In the formula (9), when the weighted summation is performed on the reality of the simulated traffic flow data corresponding to all the preset functional scenes, the weight of each preset functional scene is the same as that of the corresponding simulation traffic flow data
Figure SMS_79
The comprehensive authenticity of the simulated traffic flow data set obtained in S902 may be used as a comprehensive authenticity evaluation result for the simulated traffic flow data set.
According to the embodiment, the comprehensive authenticity of the simulated traffic flow data set is obtained by carrying out weighted summation on the authenticity of the simulated traffic flow data corresponding to all preset functional scenes, so that the comprehensive authenticity evaluation is carried out on the whole simulated traffic flow data set, the authenticity evaluation result is further enriched, and the flexibility of carrying out the authenticity evaluation on the simulated traffic flow data is improved.
Optionally, in some embodiments, when the reality of the simulated traffic flow data corresponding to all the preset functional scenes is weighted and summed, the weight of each preset functional scene may be a preset value. The weights of the partial preset functional scenes may be the same. For example, a tester may pre-configure weight values for a functional scenario according to the importance of different functional scenarios, which the disclosure is not limited to.
Optionally, in some embodiments, the reality of the simulated traffic flow data corresponding to the part of the preset functional scene may be weighted and summed to obtain the comprehensive reality of the simulated traffic flow data corresponding to the part of the preset functional scene.
In other words, the embodiment of the disclosure can acquire the reality of the simulated traffic flow data corresponding to at least two target preset functional scenes; and carrying out weighted summation on the authenticity of the simulated traffic flow data corresponding to at least two target preset functional scenes to obtain the comprehensive authenticity of the simulated traffic flow data corresponding to at least two target preset functional scenes.
Optionally, when the reality of the simulated traffic flow data corresponding to at least two target preset functional scenes is weighted and summed, the weight of each target preset functional scene is a preset value.
The simulation traffic flow data authenticity evaluation method provided by the embodiment of the disclosure is introduced through some exemplary embodiments. In order to make the implementation logic of the method for evaluating the authenticity of the simulated traffic flow data provided by the embodiment of the present disclosure more clear, the method for evaluating the authenticity of the simulated traffic flow data is further described below by way of a specific example.
Fig. 10 is a schematic flow chart of another exemplary method for evaluating the authenticity of the simulated traffic flow data according to the embodiment of the present disclosure. As shown in fig. 10, the method for evaluating the authenticity of the simulated traffic flow data provided by the embodiment of the present disclosure may include, from an overall framework: four large functional modules such as functional scene type division (or specific functional test scene type division), driving parameter extraction (or specific functional test scene initial parameter extraction), driving parameter distribution modeling (or specific functional test scene initial parameter distribution modeling), functional scene authenticity evaluation (or specific functional test scene authenticity evaluation).
The functional scene type classification module may classify a specific functional scene. The functional scenes can comprise high-speed and urban scenes, and particularly can comprise a front vehicle braking scene, a front vehicle cut-in scene, a main vehicle lane changing scene, a main vehicle overtaking scene, an intersection unprotected left turning scene and the like.
The driving parameter extraction module may extract initial parameters (i.e., driving parameters) for each specific functional scenario of the real traffic flow data and the simulated traffic flow data, respectively.
The driving parameter distribution modeling module may model the distribution of the real scene parameter data and the simulation scene parameter data, respectively, using the stream model.
The functional scene reality evaluation module can calculate the cross ratio of the real parameter probability density distribution and the simulation parameter probability density distribution of each specific functional scene so as to quantify the similarity of the two distributions and give the reality evaluation of a single specific functional test scene. Furthermore, the functional scene reality evaluation module may perform weighted summation on the similarity of each specific functional test scene to comprehensively evaluate the simulation traffic flow reality of each specific functional test scene. The specific functional test scenario is the preset functional scenario described in the foregoing embodiment.
In an exemplary embodiment, the embodiment of the present disclosure further provides a simulated traffic flow data authenticity evaluation device, which may be used to implement the simulated traffic flow data authenticity evaluation method described in the foregoing embodiment. Fig. 11 is a schematic diagram of a simulated traffic flow data authenticity evaluation device according to an embodiment of the present disclosure. As shown in fig. 11, the apparatus may include: a scene dividing unit 1101 and an authenticity evaluating unit 1102.
The scene dividing unit 1101 is configured to obtain simulated traffic flow data and real traffic flow data corresponding to a preset functional scene, where the category of the preset functional scene is divided according to at least one of a test function, a static road network structure, and a number of dynamic interactive vehicles.
The authenticity evaluation unit 1102 is configured to determine, for a target preset functional scene, an authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to a similarity between the simulated traffic flow data corresponding to the target preset functional scene and the real traffic flow data.
Illustratively, the scene division unit 1101 may be used to implement the functions of the functional scene type division module described above. The authenticity evaluation unit 1102 may be configured to implement the functions of the driving parameter extraction module, the driving parameter distribution modeling module, and the functional scene authenticity evaluation module.
For example, the reality evaluating unit 1102 may include a driving parameter extracting subunit, a driving parameter distribution creating subunit, a functional scene reality evaluating subunit, and the like.
In some possible implementations, the authenticity evaluation unit 1102 is specifically configured to: extracting simulation driving parameters in simulation traffic flow data corresponding to a target preset functional scene and real driving parameters in corresponding real traffic flow data; modeling the distribution of the simulation driving parameters and the distribution of the real driving parameters respectively through a flow model to obtain a first probability density distribution of the simulation driving parameters and a second probability density distribution of the real driving parameters; and determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the first probability density distribution and the second probability density distribution.
The similarity between the first probability density distribution and the second probability density distribution is used for representing the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene.
In some possible implementations, the encoder of the stream model implements a reversible transformation through an autoregressive neural network and a cubic spline interpolation algorithm.
In some possible implementations, the authenticity evaluation unit 1102 is specifically configured to: modeling the distribution of the simulation driving parameters through a flow model to obtain the log likelihood of the simulation driving parameters; modeling the distribution of the real driving parameters through a flow model to obtain the log likelihood of the real driving parameters; according to a preset log likelihood interval set, respectively counting the log likelihood distribution of the simulation driving parameters and the log likelihood distribution of the real driving parameters.
The log-likelihood distribution of the simulation driving parameters is used for representing the first probability density distribution, and the log-likelihood distribution of the real driving parameters is used for representing the second probability density distribution.
In some possible implementations, the authenticity evaluation unit 1102 is specifically configured to: determining a distribution area intersection ratio of the log-likelihood distribution of the simulation driving parameters and the log-likelihood distribution of the real driving parameters, wherein the distribution area intersection ratio is used for representing the similarity between the first probability density distribution and the second probability density distribution; and determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the cross-merging ratio of the distribution areas.
In some possible implementations, the authenticity assessment result includes a degree of authenticity; the authenticity evaluation unit 1102 is further configured to: obtaining the reality of simulated traffic flow data corresponding to at least two target preset functional scenes; and carrying out weighted summation on the authenticity of the simulated traffic flow data corresponding to the at least two target preset functional scenes to obtain the comprehensive authenticity of the simulated traffic flow data corresponding to the at least two target preset functional scenes.
Optionally, when the reality of the simulated traffic flow data corresponding to at least two target preset functional scenes is weighted and summed, the weight of each target preset functional scene is a preset value.
Illustratively, the simulated driving parameters and the actual driving parameters include respectively: absolute speed of the host vehicle, absolute speed of the dynamically interactive vehicle, relative distance of the host vehicle from the dynamically interactive vehicle, and relative speed.
Illustratively, the functional scenario includes at least one of: front vehicle braking scene, front vehicle cut-in scene, main vehicle lane changing scene, main vehicle overtaking scene and crossroad unprotected left turning scene.
According to an embodiment of the disclosure, the disclosure further provides an electronic device.
In an exemplary embodiment, an electronic device includes: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in the above embodiments. The electronic device may be the computer or server described above.
For example, fig. 12 is a schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 12, the electronic device 1200 includes a computing unit 1201 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) or a computer program loaded from a storage unit 1208 into a Random Access Memory (RAM). In the RAM 1203, various programs and data required for the operation of the electronic device 1200 are also available. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input/output (I/O) interface is also connected to the bus 1204.
Various components in the electronic device 1200 are connected to the I/O interface 1205, including: an input unit 1206 such as a keyboard, mouse, etc.; an output unit 1207 such as various types of displays, speakers, and the like; a storage unit 1208 such as a magnetic disk, an optical disk, or the like; and a communication unit 1209, such as a network card, modem, wireless communication transceiver, etc. The communication unit 1209 allows the electronic device 1200 to exchange information/data with other devices through a computer network, such as the internet, and/or various telecommunications networks.
The computing unit 1201 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1201 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), any suitable processor, controller, microcontroller, and the like. The computing unit 1201 performs the respective methods and processes described above, such as the simulated traffic flow data authenticity assessment method. For example, in some embodiments, the simulated traffic flow data authenticity assessment method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1208.
In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 1200 via the ROM 1202 and/or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the above-described simulated traffic flow data authenticity evaluation method may be performed.
Alternatively, in other embodiments, the computing unit 1201 may be configured to perform the simulated traffic flow data authenticity assessment method by any other suitable means (e.g. by means of firmware).
According to embodiments of the present disclosure, the present disclosure also provides a readable storage medium and a computer program product.
In an exemplary embodiment, the readable storage medium may be a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method according to the above embodiment.
In an exemplary embodiment, the computer program product comprises a computer program which, when executed by a processor, implements the method according to the above embodiments.
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed aspects are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims (21)

1. A method for evaluating the authenticity of simulated traffic flow data, the method comprising:
obtaining simulation traffic flow data and real traffic flow data corresponding to a preset functional scene, wherein the categories of the preset functional scene are divided according to at least one of test functions, static road network structures and dynamic interactive vehicle numbers;
aiming at a target preset functional scene, determining an authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene.
2. The method of claim 1, wherein the determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene comprises:
extracting simulation driving parameters in simulation traffic flow data corresponding to the target preset functional scene and real driving parameters in corresponding real traffic flow data;
modeling the distribution of the simulation driving parameters and the distribution of the real driving parameters through a flow model respectively to obtain a first probability density distribution of the simulation driving parameters and a second probability density distribution of the real driving parameters;
Determining an authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the first probability density distribution and the second probability density distribution;
the similarity between the first probability density distribution and the second probability density distribution is used for representing the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene.
3. The method of claim 2, the encoder of the stream model implementing reversible transformations by means of an autoregressive neural network and a cubic spline interpolation algorithm.
4. A method according to claim 3, wherein modeling the distribution of the simulated driving parameters and the distribution of the real driving parameters by the flow model to obtain a first probability density distribution of the simulated driving parameters and a second probability density distribution of the real driving parameters respectively includes:
modeling the distribution of the simulation driving parameters through a flow model to obtain the log likelihood of the simulation driving parameters;
modeling the distribution of the real driving parameters through a flow model to obtain the log likelihood of the real driving parameters;
Respectively counting the log-likelihood distribution of the simulation driving parameters and the log-likelihood distribution of the real driving parameters according to a preset log-likelihood interval set;
the log-likelihood distribution of the simulation driving parameters is used for representing the first probability density distribution, and the log-likelihood distribution of the real driving parameters is used for representing the second probability density distribution.
5. The method of claim 4, wherein determining the result of evaluating the authenticity of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the first probability density distribution and the second probability density distribution comprises:
determining a distribution area intersection ratio of the log-likelihood distribution of the simulated driving parameters and the log-likelihood distribution of the real driving parameters, wherein the distribution area intersection ratio is used for representing the similarity between the first probability density distribution and the second probability density distribution;
and determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the cross-merging ratio of the distribution areas.
6. The method according to any one of claims 1-5, the authenticity assessment result comprising authenticity; the method further comprises the steps of:
Obtaining the reality of simulated traffic flow data corresponding to at least two target preset functional scenes;
and carrying out weighted summation on the reality of the simulated traffic flow data corresponding to the at least two target preset functional scenes to obtain the comprehensive reality of the simulated traffic flow data corresponding to the at least two target preset functional scenes.
7. The method of claim 6, wherein when the reality of the simulated traffic flow data corresponding to the at least two target preset functional scenes is weighted and summed, the weight of each target preset functional scene is a preset value.
8. The method of any of claims 2-5, the simulated driving parameters and the actual driving parameters comprising: absolute speed of the host vehicle, absolute speed of the dynamically interactive vehicle, relative distance of the host vehicle from the dynamically interactive vehicle, and relative speed.
9. The method of any of claims 1-5, the preset functional scenario comprising at least one of: front vehicle braking scene, front vehicle cut-in scene, main vehicle lane changing scene, main vehicle overtaking scene and crossroad unprotected left turning scene.
10. An emulated traffic flow data authenticity assessment device, the device comprising:
The scene dividing unit is used for acquiring simulated traffic flow data and real traffic flow data corresponding to a preset functional scene, wherein the categories of the preset functional scene are divided according to at least one of test functions, static road network structures and dynamic interactive vehicle numbers;
the authenticity evaluation unit is used for determining an authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the simulated traffic flow data corresponding to the target preset functional scene and the real traffic flow data.
11. The apparatus of claim 10, the authenticity assessment unit being specifically configured to:
extracting simulation driving parameters in simulation traffic flow data corresponding to the target preset functional scene and real driving parameters in corresponding real traffic flow data;
modeling the distribution of the simulation driving parameters and the distribution of the real driving parameters through a flow model respectively to obtain a first probability density distribution of the simulation driving parameters and a second probability density distribution of the real driving parameters;
determining an authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the similarity between the first probability density distribution and the second probability density distribution;
The similarity between the first probability density distribution and the second probability density distribution is used for representing the similarity between the simulated traffic flow data and the real traffic flow data corresponding to the target preset functional scene.
12. The apparatus of claim 11, the encoder of the stream model implements a reversible transformation by an autoregressive neural network and a cubic spline interpolation algorithm.
13. The apparatus of claim 12, the authenticity assessment unit being specifically configured to:
modeling the distribution of the simulation driving parameters through a flow model to obtain the log likelihood of the simulation driving parameters;
modeling the distribution of the real driving parameters through a flow model to obtain the log likelihood of the real driving parameters;
respectively counting the log-likelihood distribution of the simulation driving parameters and the log-likelihood distribution of the real driving parameters according to a preset log-likelihood interval set;
the log-likelihood distribution of the simulation driving parameters is used for representing the first probability density distribution, and the log-likelihood distribution of the real driving parameters is used for representing the second probability density distribution.
14. The apparatus of claim 13, the authenticity assessment unit being specifically configured to:
Determining a distribution area intersection ratio of the log-likelihood distribution of the simulated driving parameters and the log-likelihood distribution of the real driving parameters, wherein the distribution area intersection ratio is used for representing the similarity between the first probability density distribution and the second probability density distribution;
and determining the authenticity evaluation result of the simulated traffic flow data corresponding to the target preset functional scene according to the cross-merging ratio of the distribution areas.
15. The apparatus according to any one of claims 10-14, the authenticity assessment result comprising authenticity; the authenticity evaluation unit is further configured to:
obtaining the reality of simulated traffic flow data corresponding to at least two target preset functional scenes;
and carrying out weighted summation on the reality of the simulated traffic flow data corresponding to the at least two target preset functional scenes to obtain the comprehensive reality of the simulated traffic flow data corresponding to the at least two target preset functional scenes.
16. The apparatus of claim 15, wherein the weight of each of the target preset functional scenes is a preset value when the reality of the simulated traffic flow data corresponding to the at least two target preset functional scenes is weighted and summed.
17. The apparatus of any of claims 11-14, the simulated driving parameters and the actual driving parameters comprising: absolute speed of the host vehicle, absolute speed of the dynamically interactive vehicle, relative distance of the host vehicle from the dynamically interactive vehicle, and relative speed.
18. The apparatus of any of claims 10-14, the preset functional scenario comprising at least one of: front vehicle braking scene, front vehicle cut-in scene, main vehicle lane changing scene, main vehicle overtaking scene and crossroad unprotected left turning scene.
19. An electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor;
wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.
20. A non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1-9.
21. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any of claims 1-9.
CN202310489520.2A 2023-05-04 2023-05-04 Method and device for evaluating authenticity of simulated traffic flow data Pending CN116244974A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310489520.2A CN116244974A (en) 2023-05-04 2023-05-04 Method and device for evaluating authenticity of simulated traffic flow data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310489520.2A CN116244974A (en) 2023-05-04 2023-05-04 Method and device for evaluating authenticity of simulated traffic flow data

Publications (1)

Publication Number Publication Date
CN116244974A true CN116244974A (en) 2023-06-09

Family

ID=86629777

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310489520.2A Pending CN116244974A (en) 2023-05-04 2023-05-04 Method and device for evaluating authenticity of simulated traffic flow data

Country Status (1)

Country Link
CN (1) CN116244974A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117095239A (en) * 2023-10-16 2023-11-21 光轮智能(北京)科技有限公司 Simulation asset authenticity evaluation method, control device and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120274506A1 (en) * 2011-04-28 2012-11-01 Doria David M Performance model for synthetic aperture radar automatic target recognition and method thereof
CN110782120A (en) * 2019-09-12 2020-02-11 腾讯科技(深圳)有限公司 Method, system, equipment and medium for evaluating traffic flow model
CN115114786A (en) * 2022-06-29 2022-09-27 重庆长安汽车股份有限公司 Evaluation method, system and storage medium for traffic flow simulation model
CN115981179A (en) * 2022-12-30 2023-04-18 西安深信科创信息技术有限公司 Method and device for generating test indexes of automatic driving simulation test scene

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120274506A1 (en) * 2011-04-28 2012-11-01 Doria David M Performance model for synthetic aperture radar automatic target recognition and method thereof
CN110782120A (en) * 2019-09-12 2020-02-11 腾讯科技(深圳)有限公司 Method, system, equipment and medium for evaluating traffic flow model
CN115114786A (en) * 2022-06-29 2022-09-27 重庆长安汽车股份有限公司 Evaluation method, system and storage medium for traffic flow simulation model
CN115981179A (en) * 2022-12-30 2023-04-18 西安深信科创信息技术有限公司 Method and device for generating test indexes of automatic driving simulation test scene

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
CONOR DURKAN,ET AL.: "Neural Spline Flows", 《ARXIV》, pages 119 - 16 *
中国汽车技术研究中心有限公司组 编: "《智能网联汽车测试与评价技术》", 31 January 2022, 机械工业出版社, pages: 13 - 16 *
李彪 等编: "《智能网联汽车建模与仿真技术:含实验指导》", 28 February 2023, 机械工业出版社, pages: 183 - 185 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117095239A (en) * 2023-10-16 2023-11-21 光轮智能(北京)科技有限公司 Simulation asset authenticity evaluation method, control device and storage medium
CN117095239B (en) * 2023-10-16 2023-12-15 光轮智能(北京)科技有限公司 Simulation asset authenticity evaluation method, control device and storage medium

Similar Documents

Publication Publication Date Title
CN113408141B (en) Automatic driving test method and device and electronic equipment
CN111123920A (en) Method and device for generating automatic driving simulation test scene
CN107038478B (en) Road condition prediction method and device, computer equipment and readable medium
CN109784283B (en) Remote sensing image target extraction method based on scene recognition task
CN112287982A (en) Data prediction method and device and terminal equipment
CN103699715A (en) Fluid-solid coupling method based on smoothed-particle hydrodynamics (SPH) and nonlinear finite elements
CN112559371B (en) Automatic driving test method and device and electronic equipment
CN116244974A (en) Method and device for evaluating authenticity of simulated traffic flow data
CN115616937B (en) Automatic driving simulation test method, device, equipment and computer readable medium
JP7446359B2 (en) Traffic data prediction method, traffic data prediction device, electronic equipment, storage medium, computer program product and computer program
JP2022092047A (en) Green wave speed determination method, green wave speed determination device, electronic device, storage medium, and computer program
CN113742437B (en) Map updating method, device, electronic equipment and storage medium
Lovato et al. Three-dimensional fixed-trajectory approaches to the minimum-lap time of road vehicles
CN114677653A (en) Model training method, vehicle key point detection method and corresponding devices
CN112462759B (en) Evaluation method, system and computer storage medium of rule control algorithm
CN111881121B (en) Automatic driving data filling method and device
CN109117352B (en) Server performance prediction method and device
CN114565092A (en) Neural network structure determining method and device
CN115687764B (en) Training method of vehicle track evaluation model, vehicle track evaluation method and device
CN113850297B (en) Road data monitoring method and device, electronic equipment and storage medium
CN115330067A (en) Traffic congestion prediction method and device, electronic equipment and storage medium
CN110474803B (en) Processing method of Internet of vehicles system and related equipment
Kim et al. SALT-Viz: Real-Time Visualization for Large-Scale Traffic Simulation
CN113806452B (en) Information processing method, information processing device, electronic equipment and storage medium
Luo et al. Clarifying origin-destination flows using force-directed edge bundling layout

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Country or region after: China

Address after: Room 533, 5th Floor, Building A3A4, Phase I, Anchenggu Science and Technology Park, No. 900 Wangjiang West Road, High tech Zone, Hefei City, Anhui Province, 230000

Applicant after: Anhui Xinxin Science and Technology Innovation Information Technology Co.,Ltd.

Address before: 2nd Floor, Building B2, Yunhui Valley, No. 156, Tiangu 8th Road, Software New Town, Yuhua Street Office, High-tech Zone, Xi'an City, Shaanxi Province 710000

Applicant before: Xi'an Xinxin Information Technology Co.,Ltd.

Country or region before: China