CN115543095B - Public safety digital interactive experience method and system - Google Patents

Public safety digital interactive experience method and system Download PDF

Info

Publication number
CN115543095B
CN115543095B CN202211532510.4A CN202211532510A CN115543095B CN 115543095 B CN115543095 B CN 115543095B CN 202211532510 A CN202211532510 A CN 202211532510A CN 115543095 B CN115543095 B CN 115543095B
Authority
CN
China
Prior art keywords
public safety
image
scene
experience
safety experience
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211532510.4A
Other languages
Chinese (zh)
Other versions
CN115543095A (en
Inventor
朱鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Binggu Digital Technology Co ltd
Original Assignee
Jiangsu Binggu Digital Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Binggu Digital Technology Co ltd filed Critical Jiangsu Binggu Digital Technology Co ltd
Priority to CN202211532510.4A priority Critical patent/CN115543095B/en
Publication of CN115543095A publication Critical patent/CN115543095A/en
Application granted granted Critical
Publication of CN115543095B publication Critical patent/CN115543095B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/20—Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/04—Indexing scheme for image data processing or generation, in general involving 3D image data

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Graphics (AREA)
  • Architecture (AREA)
  • Human Computer Interaction (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • Library & Information Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention relates to the technical field of digital interaction, and discloses a public safety digital interaction experience method and a public safety digital interaction experience system, wherein the method comprises the following steps: generating a public safety experience scene image by using a diffusion model based on the public safety experience scene text; reconstructing a three-dimensional public safety experience scene based on the generated public safety experience scene image; performing edge structure enhancement on a public safety experience three-dimensional scene; monitoring the position and the action of the user in real time by utilizing a user behavior monitoring algorithm; and if the situation that the user interacts with the object in the public safety experience three-dimensional scene is monitored, performing real-time interactive rendering on the interactive object. According to the invention, the automatically-adjustable and various public safety experience scene construction is realized based on the public safety experience scene text input by the user, and the public safety experience three-dimensional scene is obtained based on the three-dimensional reconstruction method, so that an experiencer can experience the simulation scene of the public safety theme more intuitively and deeply, and real-time interaction is carried out.

Description

Public safety digital interactive experience method and system
Technical Field
The invention relates to the technical field of digital interactive experience, in particular to a public safety digital interactive experience method and system.
Background
The existing public safety education method takes text teaching as a main part and improves the public safety awareness of people through modes of visiting and learning and the like, but the mode has the defects of small audience area, difficulty in popularization, high implementation cost and the like. And the traditional public safety education method lacks visual scene experience and is separated from the accident scene, so that people are difficult to pay attention to the importance of public safety precaution, and the propaganda, popularization and education promotion of public safety knowledge are restricted. Aiming at the problem, the patent provides a public safety digital interactive experience method, and public safety education is realized in a digital mode.
Disclosure of Invention
In view of the above, the invention provides a public safety digital interactive experience method, and aims to 1) realize the construction of multiple types of automatically adjustable public safety experience scenes based on public safety experience scene texts input by users, obtain public safety experience three-dimensional scenes based on a three-dimensional reconstruction method, and build a bridge between a virtual environment and a real environment, so that an experiencer can experience a simulation scene of a public safety theme more intuitively and deeply, and the problem of boring propagation of traditional public safety propaganda and education is solved; 2) The method comprises the steps of utilizing an edge structure enhancement method to carry out structure enhancement on a generated public safety experience three-dimensional scene, effectively marking objects in the scene, utilizing a user behavior monitoring algorithm to monitor the position and the action of a user in real time, and if the situation that the user and the objects in the public safety experience three-dimensional scene interact with each other is monitored, carrying out real-time interactive rendering on the interactive objects, so that the user can realize real-time interaction with the objects under the public safety theme without the help of complex wearing equipment or in the public safety scene.
In order to achieve the purpose, the public safety digital interactive experience method provided by the invention comprises the following steps:
s1: manually inputting a public safety experience scene text, and generating a public safety experience scene image by using a diffusion model based on the input text;
s2: reconstructing a three-dimensional public safety experience scene based on the generated public safety experience scene image to obtain a public safety experience three-dimensional scene;
s3: performing edge structure enhancement on a public safety experience three-dimensional scene;
s4: monitoring the position and the action of the user in real time by utilizing a user behavior monitoring algorithm;
s5: and if the situation that the user interacts with the object in the public safety experience three-dimensional scene is monitored, performing real-time interactive rendering on the interactive object.
As a further improvement of the method of the invention:
optionally, the manually inputting a public safety experience scene text in S1 includes:
the method comprises the steps that a user inputs a public safety experience scene text into a public safety digital experience system before public safety digital interactive experience, wherein the public safety experience scene text describes arrangement conditions of public safety experience scenes to be experienced and objects existing in the scenes.
Optionally, the generating, in S1, a public safety experience scene image by using a diffusion model based on the input text includes:
the method comprises the following steps of constructing a diffusion model, inputting public safety experience scene texts into the diffusion model, outputting corresponding public safety experience scene images by the diffusion model, wherein the diffusion model comprises a text feature extraction layer and an image generation layer, and the public safety experience scene images are generated by the following steps:
inputting the public safety experience scene text into a text feature extraction layer of a diffusion model, wherein the text feature extraction layer encodes the public safety experience scene text by using an independent hot method, and converts the encoded text into a text feature vector y by using an embedding method;
inputting the text feature vector y into an image generation layer, wherein the image generation layer generates a corresponding public safety experience scene image f based on the guidance of the text feature vector, and the generation formula of the public safety experience scene image is as follows:
Figure 788283DEST_PATH_IMAGE002
wherein:
Figure DEST_PATH_IMAGE003
the method comprises the steps of performing T-step sampling based on text characteristic vector f guidance on an image to be sampled in a diffusion model, and performing multi-step condition-guided sampling denoising processing on a denoised image, wherein the image which is finally successfully denoised is a public safety experience scene image.
Optionally, the training process of the diffusion model in S1 includes:
s11: constructing an image training set containing m images and describing image text feature vectors, wherein the image training set contains images of various different public safety experience scenes, and carrying out comparison on any ith reference image in the image training set
Figure 719330DEST_PATH_IMAGE004
Performing image noise diffusion in the T step;
the image noise diffusion process comprises the following steps:
Figure 614605DEST_PATH_IMAGE006
Figure 882775DEST_PATH_IMAGE008
wherein:
Figure DEST_PATH_IMAGE009
representing images
Figure 579729DEST_PATH_IMAGE004
Carrying out the process sequence of T-step image noise-adding diffusion,
Figure 275153DEST_PATH_IMAGE010
representing the result of adding noise and diffusion to the image of the t-1 step
Figure DEST_PATH_IMAGE011
Adding Gaussian noise
Figure 188882DEST_PATH_IMAGE012
Obtaining the image noise diffusion result in the t step;
Figure 260743DEST_PATH_IMAGE012
representing the Gaussian noise added in the noise adding and diffusing process of the image in the t step
Figure 45160DEST_PATH_IMAGE012
Is a Gaussian distribution
Figure DEST_PATH_IMAGE013
，
Figure 318009DEST_PATH_IMAGE014
The mean value of the gaussian distribution is represented,
Figure DEST_PATH_IMAGE015
representing the variance of the Gaussian noise added in the t step;
s12: adding a text characteristic vector of the content covered by Gaussian noise in each step of noise diffusion process to obtain an image noise diffusion process based on the text characteristic vector condition:
Figure DEST_PATH_IMAGE017
wherein:
Figure 217570DEST_PATH_IMAGE018
representing a reference image
Figure 93122DEST_PATH_IMAGE004
The feature vector of the text of (2),
Figure DEST_PATH_IMAGE019
representing text characteristic vectors of content covered by Gaussian noise added in the noise diffusion process in the t step;
s13: the sampling process of the noise-added image is the restoration process of the noise-added image, and the sampling formula of the noise-added image based on the condition is as follows:
Figure DEST_PATH_IMAGE021
Figure DEST_PATH_IMAGE023
Figure DEST_PATH_IMAGE025
wherein:
Figure 200886DEST_PATH_IMAGE026
indicates the recovery condition, selects
Figure DEST_PATH_IMAGE027
Minimum T-step noisy image
Figure 379058DEST_PATH_IMAGE028
As an image to be sampled, the image is taken,
Figure 860855DEST_PATH_IMAGE027
representing a bayesian gradient based on conditional recovery,
Figure DEST_PATH_IMAGE029
show that
Figure 200086DEST_PATH_IMAGE030
Sampling recovery into
Figure DEST_PATH_IMAGE031
The pilot parameters of (a) are set,
Figure 83728DEST_PATH_IMAGE032
representing a sampling distribution parameter conforming to normal distribution;
Figure DEST_PATH_IMAGE033
show that
Figure 432801DEST_PATH_IMAGE030
Sampling recovery into
Figure 277261DEST_PATH_IMAGE031
The formula (2).
Optionally, the reconstructing a three-dimensional public safety experience scene based on the generated public safety experience scene image in S2 includes:
carrying out three-dimensional public safety experience scene reconstruction based on the generated public safety experience scene image to obtain a public safety experience three-dimensional scene, wherein the reconstruction process of the three-dimensional public safety experience scene comprises the following steps:
s21: extracting SIFT features of the generated public safety experience scene image f to obtain K groups of feature vector sets of the public safety experience scene image f
Figure 229036DEST_PATH_IMAGE034
，
Figure DEST_PATH_IMAGE035
；
S22: constructing a three-dimensional grid image, and matching each feature vector of the public safety experience scene image to the grid vertex position of the three-dimensional grid image, wherein the side length of a three-dimensional grid in the three-dimensional grid image is
Figure 967185DEST_PATH_IMAGE036
Where v is the number of line pixels in the generated public safety experience scene image f, and the formula of the feature vector matching is:
Figure 487159DEST_PATH_IMAGE038
wherein:
Figure DEST_PATH_IMAGE039
as feature vectors
Figure 583029DEST_PATH_IMAGE040
Length of (d);
Figure DEST_PATH_IMAGE041
as feature vectors
Figure 604074DEST_PATH_IMAGE040
The coordinate position of the center of (b) on the X-axis in the two-dimensional image f;
Figure 337675DEST_PATH_IMAGE042
as feature vectors
Figure 887605DEST_PATH_IMAGE040
The coordinate position of the center of (a) in the two-dimensional image f on the Y-axis;
Figure DEST_PATH_IMAGE043
as feature vectors
Figure 706657DEST_PATH_IMAGE040
Stereo coordinates in the three-dimensional mesh image;
s23: randomly selecting a grid vertex in the three-dimensional grid image as an initial modeling point, selecting an adjacent feature vector with the minimum coordinate distance with the initial modeling point as a topological node of the initial modeling point, connecting the initial modeling point and the topological node, searching the adjacent feature vector with the minimum coordinate distance with the initial modeling point and the topological node, and connecting the three feature vectors to form an initial triangle;
s24: taking the vertex of the formed triangle as an initial modeling point, and repeating the steps until all the feature vectors in the three-dimensional grid image are connected;
s25: and for any connected triangle, rendering the pixel distribution gradient histogram of the feature vector corresponding to the vertex of the triangle into a pixel distribution result of a triangle area, and obtaining a public safety experience three-dimensional scene F.
Optionally, the performing, in S3, edge structure enhancement on the reconstructed public safety experience three-dimensional scene includes:
performing edge structure enhancement on the reconstructed public safety experience three-dimensional scene, wherein the edge structure enhancement process comprises the following steps:
s31: calculating second-order partial derivatives of the public safety experience three-dimensional scene F obtained through reconstruction in the X, Y and Z axis directions to form a partial derivative matrix of the public safety experience three-dimensional scene F
Figure 235DEST_PATH_IMAGE044
：
Figure 588342DEST_PATH_IMAGE046
Wherein:
partial derivative matrix
Figure 309173DEST_PATH_IMAGE044
The value in (A) represents the partial derivative result of the public safety experience three-dimensional scene F in any two directions;
s32: pair partial derivative matrix
Figure 740155DEST_PATH_IMAGE044
Performing characteristic decomposition to obtain three maximum characteristic values
Figure DEST_PATH_IMAGE047
；
S33: constructing three-dimensional filter coefficients
Figure 745413DEST_PATH_IMAGE048
；
S34: inputting pixel points in a three-dimensional scene of public safety experience into a filter based on a three-dimensional filter coefficient, wherein the filter formula of the filter is as follows:
Figure 188027DEST_PATH_IMAGE050
wherein:
Figure DEST_PATH_IMAGE051
representing the pixel value of any pixel point p in the three-dimensional scene of public safety experience,
Figure 17443DEST_PATH_IMAGE052
representing the result of the filtering of the corresponding pixel point p,
Figure DEST_PATH_IMAGE053
representing a pixel value threshold;
e denotes a natural constant.
Optionally, the monitoring the user position and the user action by using the user behavior monitoring algorithm in S4 includes:
monitoring the user position and the user action in real time by using a user behavior monitoring algorithm, wherein the monitoring process of the user position and the user action comprises the following steps:
s41: issuing an interactive bracelet for a user, and determining the position and posture information of the user in real time by using a position sensor and a posture sensor in the interactive bracelet, wherein the posture information of the user comprises an included angle between an arm and a body and the acceleration of the arm;
s42: calculating the distance between the real-time position of the user and the position of an article in the public safety experience three-dimensional scene, and if the distance is smaller than a preset position threshold value, indicating that the user is near the article;
s43: if the situation that the user is in the vicinity of an article in a public safety experience three-dimensional scene is monitored, inputting the posture information of the user into an interactive action recognition model, wherein the interactive action recognition model is a two-classification model, and the output result is interactive action or non-interactive action.
Optionally, in S5, when it is monitored that the user interacts with an article in the public safety experience three-dimensional scene, performing real-time interactive rendering on the interactive article, including:
presetting interaction response rules of different objects in the generated public safety experience three-dimensional scene, and performing real-time interaction rendering on interaction objects based on the interaction response rules when the interaction actions of the user and the objects in the public safety experience three-dimensional scene are monitored, wherein the interaction rendering mode comprises interaction object shape reconstruction and color reconstruction based on the interaction response rules.
In order to solve the above problems, the present invention provides a public safety digital interactive experience system, which is characterized in that the system comprises:
the image generation device is used for receiving the public safety experience scene text and generating a public safety experience scene image by using a diffusion model based on the input text;
the three-dimensional reconstruction device is used for reconstructing a three-dimensional public safety experience scene based on the generated public safety experience scene image and enhancing the edge structure of the public safety experience three-dimensional scene;
and the interaction control device is used for monitoring the position and the action of the user in real time by using a user behavior monitoring algorithm, and performing real-time interactive rendering on the interactive object if the user is monitored to perform interactive action with the object in the public safety experience three-dimensional scene.
In order to solve the above problem, the present invention also provides an electronic device, including:
a memory storing at least one instruction; and
and the processor executes the instructions stored in the memory to realize the public safety digital interactive experience method.
In order to solve the above problem, the present invention further provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the public safety digital interactive experience method.
Compared with the prior art, the invention provides a public safety digital interactive experience method, which has the following advantages:
firstly, the scheme provides an image generation method based on a public safety experience scene text, the public safety experience scene text is input into a diffusion model by constructing the diffusion model, the diffusion model outputs a corresponding public safety experience scene image, the diffusion model comprises a text feature extraction layer and an image generation layer, and the generation flow of the public safety experience scene image is as follows: inputting a public safety experience scene text into a text feature extraction layer of a diffusion model, coding the public safety experience scene text by the text feature extraction layer by using a one-hot method, and converting the coded text into a text feature vector y by using an embedding method; inputting the text feature vector y into an image generation layer, wherein the image generation layer generates a corresponding public safety experience scene image f based on the guidance of the text feature vector, and the generation formula of the public safety experience scene image is as follows:
Figure 76666DEST_PATH_IMAGE054
wherein:
Figure 712046DEST_PATH_IMAGE003
the method comprises the steps of performing T-step sampling based on text characteristic vector f guidance on an image to be sampled in a diffusion model, and performing multi-step condition-guided sampling denoising processing on a denoised image, wherein the image which is finally successfully denoised is a public safety experience scene image. The training process of the diffusion model comprises the following steps: constructing an image training set containing m images and describing image text characteristic vectors, wherein the image training set contains images of various different public safety experience scenes and aiming at any ith reference image in the image training set
Figure 773281DEST_PATH_IMAGE004
Performing image noise diffusion in the T step; the image noise diffusion process comprises the following steps:
Figure DEST_PATH_IMAGE055
Figure 242440DEST_PATH_IMAGE008
wherein:
Figure 116855DEST_PATH_IMAGE009
representing images
Figure 555926DEST_PATH_IMAGE004
Carrying out the process sequence of T-step image noise-adding diffusion,
Figure 707553DEST_PATH_IMAGE010
representing the result of adding noise and diffusion to the image of the t-1 step
Figure 206668DEST_PATH_IMAGE011
Additive gaussian noise
Figure 568379DEST_PATH_IMAGE012
Obtaining the image noise diffusion result in the t step;
Figure 420928DEST_PATH_IMAGE012
representing the Gaussian noise added in the noise adding and diffusing process of the image in the t step
Figure 82854DEST_PATH_IMAGE012
Is a Gaussian distribution
Figure 487290DEST_PATH_IMAGE013
，
Figure 707269DEST_PATH_IMAGE014
The mean value of the gaussian distribution is represented,
Figure 488143DEST_PATH_IMAGE015
representing the variance of the Gaussian noise added in the t step; adding a text characteristic vector of the content covered by Gaussian noise in each step of noise diffusion process to obtain an image noise diffusion process based on the text characteristic vector condition:
Figure 879942DEST_PATH_IMAGE056
wherein:
Figure 455279DEST_PATH_IMAGE018
representing a reference image
Figure 791583DEST_PATH_IMAGE004
The feature vector of the text of (2),
Figure 251514DEST_PATH_IMAGE019
representing text characteristic vectors of content covered by Gaussian noise added in the noise diffusion process in the t step; the sampling process of the noise-added image is the restoration process of the noise-added image, and the sampling formula of the noise-added image based on the condition is as follows:
Figure 356873DEST_PATH_IMAGE021
Figure 244058DEST_PATH_IMAGE023
Figure 67658DEST_PATH_IMAGE025
wherein:
Figure 190334DEST_PATH_IMAGE026
indicating the recovery condition, selecting
Figure 789681DEST_PATH_IMAGE027
Minimum T-step noisy image
Figure 706821DEST_PATH_IMAGE028
As an image to be sampled, a sample is taken,
Figure 17717DEST_PATH_IMAGE027
representing a bayesian gradient based on conditional recovery,
Figure 553871DEST_PATH_IMAGE029
show that
Figure 899402DEST_PATH_IMAGE030
Sampling recovery into
Figure 987444DEST_PATH_IMAGE031
The pilot parameters of (a) are,
Figure 661002DEST_PATH_IMAGE032
representing a sampling distribution parameter conforming to a normal distribution;
Figure 125481DEST_PATH_IMAGE033
show that
Figure 200885DEST_PATH_IMAGE030
Sampling is restored to
Figure 459828DEST_PATH_IMAGE031
The formula (2). According to the scheme, the constructed diffusion model is utilized, the construction of various public safety experience scenes which can be automatically adjusted is realized based on the public safety experience scene text input by the user, the public safety experience three-dimensional scene is obtained based on the three-dimensional reconstruction method, a bridge is built between the virtual environment and the real environment, an experiencer can experience the simulation scene of the public safety theme more visually and more deeply, and the problem of boring propagation of traditional public safety propaganda and education is solved.
Therefore, the scheme provides a real-time interaction method for objects in a three-dimensional scene, firstly, edge structure enhancement is carried out on a reconstructed public safety experience three-dimensional scene, and the edge structure enhancement flow is as follows: calculating second-order partial derivatives of the public safety experience three-dimensional scene F obtained through reconstruction in the X, Y and Z axis directions to form a partial derivative matrix of the public safety experience three-dimensional scene F
Figure 745315DEST_PATH_IMAGE044
：
Figure 390317DEST_PATH_IMAGE046
Wherein: partial derivative matrix
Figure 444860DEST_PATH_IMAGE044
The value in (1) represents the partial derivative result of the public safety experience three-dimensional scene F in any two directions; pair-partial derivative matrix
Figure 15650DEST_PATH_IMAGE044
Performing characteristic decomposition to obtain three maximum characteristic values
Figure 788434DEST_PATH_IMAGE047
(ii) a Constructing three-dimensional filter coefficients
Figure 594716DEST_PATH_IMAGE048
(ii) a Inputting pixel points in a public safety experience three-dimensional scene into a filter based on a three-dimensional filter coefficient, wherein the filter formula of the filter is as follows:
Figure DEST_PATH_IMAGE057
wherein:
Figure 910291DEST_PATH_IMAGE051
representing the pixel value of any pixel point p in the three-dimensional scene of public safety experience,
Figure 651982DEST_PATH_IMAGE052
representing the result of the filtering of the corresponding pixel p,
Figure 646483DEST_PATH_IMAGE053
representing a pixel value threshold; e denotes a natural constant. The method comprises the steps of monitoring the position and the action of a user in real time by using a user behavior monitoring algorithm, presetting interactive response rules of different objects in a generated public safety experience three-dimensional scene, and carrying out real-time interactive rendering on an interactive object based on the interactive response rules when the situation that the user interacts with the object in the public safety experience three-dimensional scene is monitored, wherein the interactive rendering mode comprises interactive object shape reconstruction and color reconstruction based on the interactive response rules. According to the scheme, the edge structure enhancement method is used for carrying out structure enhancement on the generated public safety experience three-dimensional scene, objects in the scene are effectively marked, the user position and the user action are monitored in real time by using a user action monitoring algorithm, and if the situation that the user and the objects in the public safety experience three-dimensional scene interact with each other is monitored, the interaction objects are subjected to structure enhancementAnd by performing real-time interactive rendering, the user can realize real-time interaction with the articles under the public safety theme without the help of complex wearable equipment or in a public safety scene.
Drawings
FIG. 1 is a schematic flowchart illustrating a public safety digital interactive experience method according to an embodiment of the present invention;
FIG. 2 is a functional block diagram of a public safety digital interactive experience system according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of an electronic device for implementing a public safety digital interactive experience method according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the application provides a public safety digital interactive experience method. The execution subject of the public safety digital interactive experience method includes, but is not limited to, at least one of electronic devices such as a server and a terminal, which can be configured to execute the method provided by the embodiments of the present application. In other words, the public safety digital interactive experience method can be executed by software or hardware installed in the terminal device or the server device, and the software can be a block chain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
Example 1:
s1: and manually inputting a public safety experience scene text, and generating a public safety experience scene image by using a diffusion model based on the input text.
The step of manually inputting the public safety experience scene text in the step S1 comprises the following steps:
according to the method, a user inputs public safety experience scene texts into a public safety digital experience system before public safety digital interactive experience, the public safety experience scene texts describe arrangement conditions of public safety experience scenes to be experienced and objects existing in the scenes, and in the embodiment of the invention, the public safety experience scenes comprise earthquake safety experience scenes, traffic safety experience scenes, city safety experience scenes, emergency rescue experience scenes and the like.
In the step S1, generating a public safety experience scene image by using a diffusion model based on the input text, including:
the method comprises the following steps of constructing a diffusion model, inputting public safety experience scene texts into the diffusion model, outputting corresponding public safety experience scene images by the diffusion model, wherein the diffusion model comprises a text feature extraction layer and an image generation layer, and the public safety experience scene images are generated by the following steps:
inputting the public safety experience scene text into a text feature extraction layer of a diffusion model, wherein the text feature extraction layer encodes the public safety experience scene text by using an independent hot method, and converts the encoded text into a text feature vector y by using an embedding method;
inputting the text feature vector y into an image generation layer, wherein the image generation layer generates a corresponding public safety experience scene image f based on the guidance of the text feature vector, and the generation formula of the public safety experience scene image is as follows:
Figure 256456DEST_PATH_IMAGE002
wherein:
Figure 393914DEST_PATH_IMAGE003
the method comprises the steps of performing T-step sampling guided by a text characteristic vector f on an image to be sampled in a diffusion model, and performing multi-step condition guided sampling denoising processing on a denoised image, wherein the image which is finally successfully denoised is a public safety experience scene image.
The training process of the diffusion model in the S1 comprises the following steps:
s11: constructing an image training set containing m images and describing image text feature vectorsThe image training set comprises images of various different public safety experience scenes, and any ith reference image in the image training set
Figure 431140DEST_PATH_IMAGE004
Performing image noise diffusion in the T step;
the image noise diffusion process comprises the following steps:
Figure 788303DEST_PATH_IMAGE058
Figure 936388DEST_PATH_IMAGE008
wherein:
Figure 820030DEST_PATH_IMAGE009
representing images
Figure 903524DEST_PATH_IMAGE004
Carrying out the process sequence of T-step image noise-adding diffusion,
Figure 872617DEST_PATH_IMAGE010
representing the result of adding noise and diffusion to the image of the t-1 step
Figure 824392DEST_PATH_IMAGE011
Adding Gaussian noise
Figure 437907DEST_PATH_IMAGE012
Obtaining the image noise diffusion result in the t step;
Figure 816936DEST_PATH_IMAGE012
representing the Gaussian noise added in the t-step image noise adding diffusion process, wherein the Gaussian noise is
Figure 7746DEST_PATH_IMAGE012
Is a Gaussian distribution
Figure 134184DEST_PATH_IMAGE013
，
Figure 726839DEST_PATH_IMAGE014
The mean value of the gaussian distribution is represented,
Figure 152135DEST_PATH_IMAGE015
representing the variance of the Gaussian noise added in the t step;
s12: adding a text characteristic vector of the content covered by Gaussian noise in each step of noise diffusion process to obtain an image noise diffusion process based on the text characteristic vector condition:
Figure 830242DEST_PATH_IMAGE056
wherein:
Figure 389399DEST_PATH_IMAGE018
representing a reference image
Figure 711927DEST_PATH_IMAGE004
The feature vector of the text of (2),
Figure 698337DEST_PATH_IMAGE019
representing text characteristic vectors of content covered by Gaussian noise added in the noise diffusion process in the t step;
s13: the sampling process of the noise-added image is the restoration process of the noise-added image, and the sampling formula of the noise-added image based on the condition is as follows:
Figure 739106DEST_PATH_IMAGE021
Figure 836375DEST_PATH_IMAGE023
Figure 403622DEST_PATH_IMAGE025
wherein:
Figure 934836DEST_PATH_IMAGE026
indicating the recovery condition, selecting
Figure 587534DEST_PATH_IMAGE027
Minimum T-step noisy image
Figure 98281DEST_PATH_IMAGE028
As an image to be sampled, the image is taken,
Figure 51193DEST_PATH_IMAGE027
representing a bayesian gradient based on conditional recovery,
Figure 989193DEST_PATH_IMAGE029
show that
Figure 129188DEST_PATH_IMAGE030
Sampling recovery into
Figure 443625DEST_PATH_IMAGE031
The pilot parameters of (a) are,
Figure 719886DEST_PATH_IMAGE032
representing a sampling distribution parameter conforming to a normal distribution;
Figure 219000DEST_PATH_IMAGE033
show that
Figure 488701DEST_PATH_IMAGE030
Sampling recovery into
Figure 465884DEST_PATH_IMAGE031
The formula (2).
S2: and reconstructing a three-dimensional public safety experience scene based on the generated public safety experience scene image to obtain a public safety experience three-dimensional scene.
In the step S2, reconstructing a three-dimensional public safety experience scene based on the generated public safety experience scene image includes:
carrying out three-dimensional public safety experience scene reconstruction based on the generated public safety experience scene image to obtain a public safety experience three-dimensional scene, wherein the reconstruction process of the three-dimensional public safety experience scene comprises the following steps:
s21: extracting SIFT features of the generated public safety experience scene image f to obtain K groups of feature vector sets of the public safety experience scene image f
Figure 737597DEST_PATH_IMAGE034
，
Figure 407613DEST_PATH_IMAGE035
；
S22: constructing a three-dimensional grid image, and matching each feature vector of the public safety experience scene image to the grid vertex position of the three-dimensional grid image, wherein the side length of a three-dimensional grid in the three-dimensional grid image is
Figure 131986DEST_PATH_IMAGE036
Where v is the number of line pixels in the generated public safety experience scene image f, and the formula of the feature vector matching is:
Figure 178440DEST_PATH_IMAGE038
wherein:
Figure 304659DEST_PATH_IMAGE039
as feature vectors
Figure 879996DEST_PATH_IMAGE040
Length of (d);
Figure 481879DEST_PATH_IMAGE041
as feature vectors
Figure 440346DEST_PATH_IMAGE040
The coordinate position of the center of (b) on the X-axis in the two-dimensional image f;
Figure 545705DEST_PATH_IMAGE042
as feature vectors
Figure 432889DEST_PATH_IMAGE040
The coordinate position of the center of (a) in the two-dimensional image f on the Y-axis;
Figure 522068DEST_PATH_IMAGE043
as feature vectors
Figure 520111DEST_PATH_IMAGE040
Stereo coordinates in the three-dimensional mesh image;
s23: randomly selecting a grid vertex in the three-dimensional grid image as an initial modeling point, selecting an adjacent feature vector with the minimum coordinate distance with the initial modeling point as a topological node of the initial modeling point, connecting the initial modeling point and the topological node, searching the adjacent feature vector with the minimum coordinate distance with the initial modeling point and the topological node, and connecting the three feature vectors to form an initial triangle;
s24: taking the vertex of the formed triangle as an initial modeling point, and repeating the steps until all the feature vectors in the three-dimensional grid image are connected;
s25: and for any connected triangle, rendering the pixel distribution gradient histogram of the feature vector corresponding to the vertex of the triangle into a pixel distribution result of a triangle area to obtain a public safety experience three-dimensional scene F.
S3: and performing edge structure enhancement on the three-dimensional scene of public safety experience.
And in the S3, performing edge structure enhancement on the reconstructed public safety experience three-dimensional scene, including:
performing edge structure enhancement on the reconstructed public safety experience three-dimensional scene, wherein the edge structure enhancement process comprises the following steps:
s31: calculating second-order partial derivatives of the public safety experience three-dimensional scene F obtained through reconstruction in the X, Y and Z axis directions to form a partial derivative matrix of the public safety experience three-dimensional scene F
Figure 745556DEST_PATH_IMAGE044
：
Figure DEST_PATH_IMAGE059
Wherein:
partial derivative matrix
Figure 334800DEST_PATH_IMAGE044
The value in (1) represents the partial derivative result of the public safety experience three-dimensional scene F in any two directions;
s32: pair partial derivative matrix
Figure 645696DEST_PATH_IMAGE044
Performing characteristic decomposition to obtain three maximum characteristic values
Figure 943035DEST_PATH_IMAGE047
；
S33: constructing three-dimensional filter coefficients
Figure 288566DEST_PATH_IMAGE048
；
S34: inputting pixel points in a public safety experience three-dimensional scene into a filter based on a three-dimensional filter coefficient, wherein the filter formula of the filter is as follows:
Figure 251974DEST_PATH_IMAGE050
wherein:
Figure 50166DEST_PATH_IMAGE051
representing the pixel value of any pixel point p in the three-dimensional scene of public safety experience,
Figure 655591DEST_PATH_IMAGE052
representing the result of the filtering of the corresponding pixel point p,
Figure 855628DEST_PATH_IMAGE053
representing a pixel value threshold;
e denotes a natural constant.
S4: and monitoring the position and the action of the user in real time by using a user behavior monitoring algorithm.
In the step S4, monitoring the user position and the user action by using a user behavior monitoring algorithm includes:
monitoring the user position and the user action in real time by using a user behavior monitoring algorithm, wherein the monitoring process of the user position and the user action comprises the following steps:
s41: the method comprises the steps that an interactive bracelet is issued for a user, and the position and posture information of the user is determined in real time by utilizing a position sensor and a posture sensor in the interactive bracelet, wherein the posture information of the user comprises an included angle between an arm and a body and the acceleration of the arm;
s42: calculating the distance between the real-time position of the user and the position of an article in the public safety experience three-dimensional scene, and if the distance is smaller than a preset position threshold value, indicating that the user is near the article;
s43: if the situation that the user is in the vicinity of an article in a public safety experience three-dimensional scene is monitored, inputting the posture information of the user into an interactive action recognition model, wherein the interactive action recognition model is a two-classification model, and the output result is interactive action or non-interactive action.
S5: and if the situation that the user interacts with the object in the public safety experience three-dimensional scene is monitored, performing real-time interactive rendering on the interactive object.
In the step S5, when it is monitored that the user interacts with an article in the public safety experience three-dimensional scene, performing real-time interactive rendering on the interactive article, including:
presetting interaction response rules of different objects in the generated public safety experience three-dimensional scene, and performing real-time interaction rendering on interaction objects based on the interaction response rules when the interaction actions of the user and the objects in the public safety experience three-dimensional scene are monitored, wherein the interaction rendering mode comprises interaction object shape reconstruction and color reconstruction based on the interaction response rules.
Example 2:
fig. 2 is a functional block diagram of a public safety digital interactive experience system according to an embodiment of the present invention, which can implement the public safety digital interactive experience method in embodiment 1.
The public safety digital interactive experience system 100 of the present invention can be installed in an electronic device. According to the realized functions, the public safety digital interactive experience system can comprise an image generation device 101, a three-dimensional reconstruction device 102 and an interactive control device 103. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and that can perform a fixed function, and that are stored in a memory of the electronic device.
The image generation device 101 is used for receiving the public safety experience scene text and generating a public safety experience scene image by using a diffusion model based on the input text;
the three-dimensional reconstruction device 102 is used for reconstructing a three-dimensional public safety experience scene based on the generated public safety experience scene image and enhancing the edge structure of the public safety experience three-dimensional scene;
and the interaction control device 103 is used for monitoring the position and the action of the user in real time by using a user behavior monitoring algorithm, and performing real-time interactive rendering on the interactive object if the user is monitored to perform interactive action with the object in the public safety experience three-dimensional scene.
In detail, when the modules in the public safety digital interactive experience system 100 according to the embodiment of the present invention are used, the same technical means as the public safety digital interactive experience method described in fig. 1 are adopted, and the same technical effects can be produced, which is not described herein again.
Example 3:
fig. 3 is a schematic structural diagram of an electronic device for implementing a public safety digital interactive experience method according to an embodiment of the present invention.
The electronic device 1 may comprise a processor 10, a memory 11, a communication interface 13 and a bus, and may further comprise a computer program, such as a program 12, stored in the memory 11 and executable on the processor 10.
The memory 11 includes at least one type of readable storage medium, which includes flash memory, removable hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, e.g. a removable hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used not only to store application software installed in the electronic device 1 and various types of data, such as codes of the program 12, but also to temporarily store data that has been output or will be output.
The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 10 is a Control Unit of the electronic device, connects various components of the whole electronic device by using various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (programs 12 for implementing digital interactive experiences, etc.) stored in the memory 11 and calling data stored in the memory 11.
The communication interface 13 may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), and is generally used to establish a communication connection between the electronic device 1 and other electronic devices and to implement connection communication between internal components of the electronic devices.
The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
Fig. 3 shows only an electronic device with components, and it will be understood by those skilled in the art that the structure shown in fig. 3 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than those shown, or some components may be combined, or a different arrangement of components.
For example, although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The program 12 stored in the memory 11 of the electronic device 1 is a combination of instructions that, when executed in the processor 10, enable:
receiving a public safety experience scene text, and generating a public safety experience scene image by using a diffusion model based on the text;
performing three-dimensional public safety experience scene reconstruction based on the generated public safety experience scene image to obtain a public safety experience three-dimensional scene;
performing edge structure enhancement on a public safety experience three-dimensional scene;
monitoring the position and the action of a user in real time by using a user behavior monitoring algorithm;
and if the situation that the user interacts with the object in the public safety experience three-dimensional scene is monitored, performing real-time interactive rendering on the interactive object.
Specifically, the specific implementation method of the processor 10 for the instruction may refer to the description of the relevant steps in the embodiments corresponding to fig. 1 to fig. 3, which is not repeated herein.
It should be noted that the above-mentioned numbers of the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments. And the terms "comprises," "comprising," or any other variation thereof, herein are intended to cover a non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of another identical element in a process, apparatus, article, or method comprising the element.
Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) as described above and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.
The above description is only a preferred embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes, which are made by using the contents of the present specification and the accompanying drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.

Claims (5)

1. A method for public safety digital interactive experience, the method comprising:
s1: manually inputting a public safety experience scene text, and generating a public safety experience scene image by using a diffusion model based on the input text;
s2: reconstructing a three-dimensional public safety experience scene based on the generated public safety experience scene image to obtain a public safety experience three-dimensional scene;
s3: carry out edge structure enhancement to public safety experience three-dimensional scene, include:
performing edge structure enhancement on the reconstructed public safety experience three-dimensional scene, wherein the edge structure enhancement process comprises the following steps:
s31: calculating second-order partial derivatives of the public safety experience three-dimensional scene F obtained through reconstruction in the X, Y and Z axis directions to form a partial derivative matrix of the public safety experience three-dimensional scene F
Figure QLYQS_1
：
Figure QLYQS_2
Wherein:
the value in the partial derivative matrix represents the partial derivative result of the public safety experience three-dimensional scene F in any two directions;
s32: pair partial derivative matrix
Figure QLYQS_3
A characteristic decomposition is carried out, the decomposition results in the maximum three characteristic values->
Figure QLYQS_4
；
S33: constructing three-dimensional filter coefficients
Figure QLYQS_5
；
S34: inputting pixel points in a three-dimensional scene of public safety experience into a filter based on a three-dimensional filter coefficient, wherein the filter formula of the filter is as follows:
Figure QLYQS_6
wherein:
Figure QLYQS_7
represents the pixel value of any pixel point p in the three-dimensional scene of public safety experience>
Figure QLYQS_8
Represents the filtering result of the corresponding pixel point p, and->
Figure QLYQS_9
Representing a pixel value threshold;
e represents a natural constant;
s4: monitoring the position and the action of the user in real time by utilizing a user behavior monitoring algorithm;
s5: if the situation that the user interacts with the object in the public safety experience three-dimensional scene is monitored, performing real-time interactive rendering on the interactive object;
wherein, the step of manually inputting the public safety experience scene text in the step S1 comprises the following steps:
a user inputs a public safety experience scene text into a public safety digital experience system before public safety digital interactive experience, wherein the public safety experience scene text describes arrangement conditions of public safety experience scenes to be experienced and articles existing in the scenes;
in S1, generating a public safety experience scene image by using a diffusion model based on an input text includes:
the method comprises the following steps of constructing a diffusion model, inputting public safety experience scene texts into the diffusion model, outputting corresponding public safety experience scene images by the diffusion model, wherein the diffusion model comprises a text feature extraction layer and an image generation layer, and the public safety experience scene images are generated by the following steps:
inputting the public safety experience scene text into a text feature extraction layer of a diffusion model, wherein the text feature extraction layer encodes the public safety experience scene text by using an independent hot method, and converts the encoded text into a text feature vector y by using an embedding method;
inputting the text feature vector y into an image generation layer, wherein the image generation layer generates a corresponding public safety experience scene image f based on the guidance of the text feature vector, and the generation formula of the public safety experience scene image is as follows:
Figure QLYQS_10
wherein:
Figure QLYQS_11
representing T-step sampling conducted on an image to be sampled in the diffusion model based on the guidance of a text feature vector f;
wherein the training process of the diffusion model in S1 includes:
s11: constructing an image training set containing m images and describing image text feature vectors, wherein the image training set contains images of various different public safety experience scenes, and carrying out comparison on any ith reference image in the image training set
Figure QLYQS_14
Performing image noise diffusion in the T step; the image noise diffusion process comprises the following steps: />
Figure QLYQS_16
，
Figure QLYQS_18
Wherein: />
Figure QLYQS_12
Representing pairs of images>
Figure QLYQS_15
Carries out the process sequence of T-step image plus noise diffusion and then judges>
Figure QLYQS_17
Means for adding a noise diffusion result to the image of the t-1 th step>
Figure QLYQS_19
Additive Gaussian noise>
Figure QLYQS_13
Obtaining the image noise diffusion result in the t step;
Figure QLYQS_20
representing the Gaussian noise added in the t-th image plus noise diffusion process, and the Gaussian noise->
Figure QLYQS_21
Is a Gaussian distribution->
Figure QLYQS_22
，/>
Figure QLYQS_23
Mean value representing a Gaussian distribution>
Figure QLYQS_24
Representing the variance of the Gaussian noise added in the t step;
s12: adding a text characteristic vector of the content covered by Gaussian noise in each step of noise diffusion process to obtain an image noise diffusion process based on the text characteristic vector condition:
Figure QLYQS_25
wherein:
Figure QLYQS_26
represents a reference image pick>
Figure QLYQS_27
Is selected based on the text feature vector of (4), and>
Figure QLYQS_28
representing text characteristic vectors of content covered by Gaussian noise added in the noise diffusion process in the t step;
s13: the sampling process of the noise-added image is the restoration process of the noise-added image, and the sampling formula of the noise-added image based on the condition is as follows:
Figure QLYQS_29
，/>
Figure QLYQS_30
，
Figure QLYQS_31
wherein:
Figure QLYQS_34
indicates a recovery condition and selects->
Figure QLYQS_37
Minimum T-step noisy image>
Figure QLYQS_39
As an image to be sampled, a sample is taken,
Figure QLYQS_33
representing a Bayesian gradient based upon condition recovery, greater than or equal to>
Figure QLYQS_35
Indicates will->
Figure QLYQS_36
The sample is restored to>
Figure QLYQS_38
Is taken over by the base station>
Figure QLYQS_32
Representing a sampling distribution parameter conforming to a normal distribution;
Figure QLYQS_40
indicates will->
Figure QLYQS_41
Sample is restored to->
Figure QLYQS_42
Is disclosedFormula (II) is shown.
2. The public safety digital interactive experience method as claimed in claim 1, wherein the S2, performing three-dimensional public safety experience scene reconstruction based on the generated public safety experience scene image, comprises:
carrying out three-dimensional public safety experience scene reconstruction based on the generated public safety experience scene image to obtain a public safety experience three-dimensional scene, wherein the reconstruction process of the three-dimensional public safety experience scene comprises the following steps:
s21: extracting SIFT features of the generated public safety experience scene image f to obtain K groups of feature vector sets of the public safety experience scene image f
Figure QLYQS_43
，/>
Figure QLYQS_44
；
S22: constructing a three-dimensional grid image, and matching each feature vector of the public safety experience scene image to the grid vertex position of the three-dimensional grid image, wherein the side length of a three-dimensional grid in the three-dimensional grid image is
Figure QLYQS_45
Where v is the number of line pixels in the generated public safety experience scene image f, and the formula for matching the feature vectors is: />
Figure QLYQS_46
Wherein:
Figure QLYQS_47
is a feature vector->
Figure QLYQS_48
Length of (d);
Figure QLYQS_49
is a feature vector->
Figure QLYQS_50
The coordinate position of the center of (b) on the X-axis in the two-dimensional image f;
Figure QLYQS_51
is a feature vector->
Figure QLYQS_52
The coordinate position of the center of (a) in the two-dimensional image f on the Y-axis;
Figure QLYQS_53
feature vector->
Figure QLYQS_54
Stereo coordinates in the three-dimensional mesh image;
s23: randomly selecting a grid vertex in the three-dimensional grid image as an initial modeling point, selecting an adjacent feature vector with the minimum coordinate distance with the initial modeling point as a topological node of the initial modeling point, connecting the initial modeling point and the topological node, searching the adjacent feature vector with the minimum coordinate distance with the initial modeling point and the topological node, and connecting the three feature vectors to form an initial triangle;
s24: taking the vertex of the formed triangle as an initial modeling point, and repeating the steps until all the feature vectors in the three-dimensional grid image are connected;
s25: and for any connected triangle, rendering the pixel distribution gradient histogram of the feature vector corresponding to the vertex of the triangle into a pixel distribution result of a triangle area, and obtaining a public safety experience three-dimensional scene F.
3. A public safety digital interactive experience method as claimed in claim 1, wherein said monitoring user location and user actions in S4 by using a user behavior monitoring algorithm comprises:
monitoring the user position and the user action in real time by using a user behavior monitoring algorithm, wherein the monitoring process of the user position and the user action comprises the following steps:
s41: the method comprises the steps that an interactive bracelet is issued for a user, and the position and posture information of the user is determined in real time by utilizing a position sensor and a posture sensor in the interactive bracelet, wherein the posture information of the user comprises an included angle between an arm and a body and the acceleration of the arm;
s42: calculating the distance between the real-time position of the user and the position of an article in the public safety experience three-dimensional scene, and if the distance is smaller than a preset position threshold value, indicating that the user is near the article;
s43: if the situation that the user is in the vicinity of an article in the three-dimensional scene of public safety experience is monitored, inputting the posture information of the user into an interactive action recognition model, wherein the interactive action recognition model is a two-class model, and outputting the result that interactive action occurs or no interactive action occurs.
4. The public safety digital interactive experience method as claimed in claim 3, wherein in step S5, when it is monitored that the user interacts with an article in the three-dimensional public safety experience scene, the real-time interactive rendering is performed on the interactive article, and the method comprises the following steps:
presetting interaction response rules of different objects in the generated public safety experience three-dimensional scene, and performing real-time interaction rendering on interaction objects based on the interaction response rules when the interaction actions of the user and the objects in the public safety experience three-dimensional scene are monitored, wherein the interaction rendering mode comprises interaction object shape reconstruction and color reconstruction based on the interaction response rules.
5. A public safety digital interactive experience system, the system comprising:
the image generation device is used for receiving the public safety experience scene text and generating a public safety experience scene image by using a diffusion model based on the input text;
the three-dimensional reconstruction device is used for reconstructing a three-dimensional public safety experience scene based on the generated public safety experience scene image and enhancing the edge structure of the public safety experience three-dimensional scene;
the interactive control device is used for monitoring the position and the action of a user in real time by using a user behavior monitoring algorithm, and if the user is monitored to interact with an article in a public safety experience three-dimensional scene, performing real-time interactive rendering on the interactive article to realize the public safety digital interactive experience method as claimed in any one of claims 1 to 4.
CN202211532510.4A 2022-12-02 2022-12-02 Public safety digital interactive experience method and system Active CN115543095B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211532510.4A CN115543095B (en) 2022-12-02 2022-12-02 Public safety digital interactive experience method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211532510.4A CN115543095B (en) 2022-12-02 2022-12-02 Public safety digital interactive experience method and system

Publications (2)

Publication Number Publication Date
CN115543095A CN115543095A (en) 2022-12-30
CN115543095B true CN115543095B (en) 2023-04-11

Family

ID=84721667

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211532510.4A Active CN115543095B (en) 2022-12-02 2022-12-02 Public safety digital interactive experience method and system

Country Status (1)

Country Link
CN (1) CN115543095B (en)

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108919954B (en) * 2018-06-29 2021-03-23 蓝色智库（北京）科技发展有限公司 Dynamic change scene virtual and real object collision interaction method
CN114399595A (en) * 2021-12-29 2022-04-26 江苏华实广告有限公司 Automatic image processing method, system and terminal for three-dimensional panoramic digital exhibition hall
CN114917590B (en) * 2022-07-21 2022-12-09 广州华立科技股份有限公司 Virtual reality game system

Also Published As

Publication number Publication date
CN115543095A (en) 2022-12-30

Similar Documents

Publication Publication Date Title
CN109902659B (en) Method and apparatus for processing human body image
CN110097086A (en) Image generates model training method, image generating method, device, equipment and storage medium
CN112836064A (en) Knowledge graph complementing method and device, storage medium and electronic equipment
CN111047509A (en) Image special effect processing method and device and terminal
CN112131985A (en) Real-time light human body posture estimation method based on OpenPose improvement
CN112381707B (en) Image generation method, device, equipment and storage medium
CN114241558B (en) Model training method, video generating method and device, equipment and medium
CN114004905B (en) Method, device, equipment and storage medium for generating character style pictogram
JP7282474B2 (en) Encryption mask determination method, encryption mask determination device, electronic device, storage medium, and computer program
CN116309983B (en) Training method and generating method and device of virtual character model and electronic equipment
CN111274937A (en) Fall detection method and device, electronic equipment and computer-readable storage medium
CN116778527A (en) Human body model construction method, device, equipment and storage medium
CN112699857A (en) Living body verification method and device based on human face posture and electronic equipment
CN112714263B (en) Video generation method, device, equipment and storage medium
CN112669431B (en) Image processing method, apparatus, device, storage medium, and program product
CN115543095B (en) Public safety digital interactive experience method and system
CN115222895B (en) Image generation method, device, equipment and storage medium
CN116524162A (en) Three-dimensional virtual image migration method, model updating method and related equipment
CN113627394B (en) Face extraction method and device, electronic equipment and readable storage medium
CN112507903B (en) False face detection method, false face detection device, electronic equipment and computer readable storage medium
CN115035219A (en) Expression generation method and device and expression generation model training method and device
CN113792671A (en) Method and device for detecting face synthetic image, electronic equipment and medium
CN112132871A (en) Visual feature point tracking method and device based on feature optical flow information, storage medium and terminal
CN114821216A (en) Method for modeling and using picture descreening neural network model and related equipment
CN111582208A (en) Method and device for generating organism posture key point information

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant