CN117707590A - Continuous integrated state tracking method, device, equipment and medium for storage software - Google Patents
Continuous integrated state tracking method, device, equipment and medium for storage software Download PDFInfo
- Publication number
- CN117707590A CN117707590A CN202311778124.8A CN202311778124A CN117707590A CN 117707590 A CN117707590 A CN 117707590A CN 202311778124 A CN202311778124 A CN 202311778124A CN 117707590 A CN117707590 A CN 117707590A
- Authority
- CN
- China
- Prior art keywords
- result
- compiling
- tool
- static scanning
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 71
- 230000003068 static effect Effects 0.000 claims abstract description 160
- 238000012360 testing method Methods 0.000 claims abstract description 140
- 230000010354 integration Effects 0.000 claims abstract description 28
- 238000010276 construction Methods 0.000 claims description 24
- 230000009193 crawling Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 10
- XSQUKJJJFZCRTK-NJFSPNSNSA-N UREA C 14 Chemical compound N[14C](N)=O XSQUKJJJFZCRTK-NJFSPNSNSA-N 0.000 claims description 8
- 238000011161 development Methods 0.000 abstract description 28
- 230000008569 process Effects 0.000 abstract description 26
- 238000004458 analytical method Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 8
- 238000007689 inspection Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 230000001960 triggered effect Effects 0.000 description 5
- 238000011144 upstream manufacturing Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 230000000977 initiatory effect Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 239000000047 product Substances 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012384 transportation and delivery Methods 0.000 description 2
- 241000258241 Mantis Species 0.000 description 1
- 241001522316 Pyrrhula pyrrhula Species 0.000 description 1
- 238000012550 audit Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000007795 chemical reaction product Substances 0.000 description 1
- 230000002354 daily effect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000011990 functional testing Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000002688 persistence Effects 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Stored Programmes (AREA)
Abstract
The application discloses a continuous integrated state tracking method, a device, an electronic device and a computer readable storage medium for storage software, which are applied to a jenkins tool deployed on a centos system and comprise the following steps: scanning the execution codes of the storage software according to the tracking command to obtain a static scanning result, wherein the static scanning result comprises an item updating code and a personal yield code; utilizing a preset compiling tool to build a compiling project for the static scanning result to obtain a compiling result, wherein the compiling result comprises a compiling result of each functional module and an integral compiling result; when the compiling engineering is successfully constructed, deploying and executing each test case by using a preset compiling tool to obtain a corresponding test result; obtaining vulnerability information of each result by using a preset crawler script; and outputting the results and the vulnerability information to a log file as a continuous integrated state tracking result of the storage software. The scheme can track the continuous integration state of the storage software in the development process in real time, so that the development efficiency and version quality of the storage software are improved.
Description
Technical Field
The present invention relates to the field of software development technologies, and in particular, to a method for tracking a continuous integrated state of stored software, and also relates to a device for tracking a continuous integrated state of stored software, an electronic device, and a computer readable storage medium.
Background
Today, in the age of the rapid development of the internet industry, the development, delivery frequency and period of software products are getting shorter and shorter. The requirements of software development on development efficiency and product quality are higher and higher, and the purpose of continuous integration is to enable the product to iterate rapidly, and meanwhile, the high quality can be maintained. However, at present, for the development of a brand new software product, a storage software continuous integration state tracking report system is not completely standardized, and especially for new products, it takes longer time to build a storage software continuous integration state tracking report system from scratch, so that the development efficiency of the storage software and the version quality of the storage software are reduced.
Therefore, how to track the continuous integration state of the storage software in the development process in real time so as to effectively improve the development efficiency of the storage software and the version quality thereof is a problem to be solved by the technicians in the field.
Disclosure of Invention
The purpose of the application is to provide a continuous integrated state tracking method of storage software, which can track the continuous integrated state of the storage software in the development process in real time, and effectively improve the development efficiency and version quality of the storage software; another object of the present application is to provide a continuously integrated state tracking device, an electronic device and a computer readable storage medium for storing software, which have the above advantages.
In a first aspect, the present application provides a method for continuously integrated state tracking of stored software, applied to a jenkins tool deployed on a centos system, comprising:
performing static scanning on the execution code of the storage software according to the tracking command to obtain a static scanning result; the static scanning result comprises an item updating code and a personal yield code;
constructing a compiling project for the static scanning result by using a preset compiling tool to obtain a compiling result; the compiling result comprises a compiling result of each functional module and an overall compiling result;
when the compiling engineering is successfully constructed, deploying and executing each test case by using a preset compiling tool to obtain a test result of each test case;
Obtaining vulnerability information of the static scanning result, the compiling result and the testing result by using a preset crawler script;
and taking the static scanning result, the compiling result, the testing result and the vulnerability information as continuous integration state tracking results of the storage software, and outputting the continuous integration state tracking results to a log file.
Optionally, the static scanning is performed on the execution code of the storage software according to the tracking command to obtain a static scanning result, including:
and starting a coverage tool or a Fortify tool or a Pc-link tool according to the tracking command, and performing static scanning on an execution code of the storage software to obtain a static scanning result.
Optionally, the static scanning is performed on the execution code of the storage software according to the tracking command to obtain a static scanning result, including:
pulling the execution code in the storage software according to the tracking command, wherein the execution code is the update code of the current tracking task compared with the last tracking task;
and scanning and analyzing the execution code by using a git command to obtain the static scanning result.
Optionally, the constructing a compiling project for the static scan result by using a preset compiling tool to obtain a compiling result includes:
And constructing the compiling engineering for the static scanning result by using an ant tool or a maven tool or a makefile tool or a make tool to obtain the compiling result.
Optionally, the deploying and executing each test case by using a preset writing tool, to obtain a test result of each test case, includes:
and deploying and executing each test case by using a pytest tool to obtain a test result of each test case.
Optionally, the obtaining vulnerability information of the static scan result, the compiling result, and the testing result by using a preset crawler script includes:
crawling preset key rows in the static scanning result by using the preset crawler script to obtain vulnerability information of the static scanning result and file path information corresponding to the vulnerability information;
crawling the compiling results of each functional module by using the preset crawler script, and obtaining failure compiling information corresponding to the compiling results of the functional modules as vulnerability information of the compiling results;
and crawling the test result by using the preset crawler script to obtain a test case with failed test as vulnerability information of the test result.
Optionally, the continuous integrated state tracking method of the storage software further includes:
When the compiling engineering construction fails, utilizing the preset crawler script to acquire vulnerability information of the static scanning result;
and taking the static scanning result, vulnerability information of the static scanning result and the compiling result as continuous integrated state tracking results of the storage software, and outputting the continuous integrated state tracking results to the log file.
In a second aspect, the present application also discloses a continuously integrated state tracking device storing software, applied to a jenkins tool deployed on a centos system, comprising:
the scanning module is used for carrying out static scanning on the execution codes of the storage software according to the tracking command to obtain a static scanning result; the static scanning result comprises an item updating code and a personal yield code;
the compiling module is used for constructing compiling engineering for the static scanning result by utilizing a preset compiling tool to obtain a compiling result; the compiling result comprises a single-module compiling result and an overall compiling result;
the execution module is used for deploying and executing each test case by using a preset writing tool when the compiling engineering is successfully constructed, and obtaining the test result of each test case;
the crawling module is used for acquiring vulnerability information of the static scanning result, the compiling result and the testing result by utilizing a preset crawler script;
And the output module is used for taking the static scanning result, the compiling result, the testing result and the vulnerability information as continuous integrated state tracking results of the storage software and outputting the continuous integrated state tracking results to a log file.
In a third aspect, the present application also discloses an electronic device, including:
a memory for storing a computer program;
a processor for implementing the steps of any of the continuous integrated state tracking methods of stored software described above when executing the computer program.
In a fourth aspect, the present application also discloses a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any of the continuous integrated state tracking methods of stored software described above.
The application provides a continuous integrated state tracking method of storage software, which is applied to a jenkins tool deployed on a centos system and comprises the following steps: performing static scanning on the execution code of the storage software according to the tracking command to obtain a static scanning result; the static scanning result comprises an item updating code and a personal yield code; constructing a compiling project for the static scanning result by using a preset compiling tool to obtain a compiling result; the compiling result comprises a compiling result of each functional module and an overall compiling result; when the compiling engineering is successfully constructed, deploying and executing each test case by using a preset compiling tool to obtain a test result of each test case; obtaining vulnerability information of the static scanning result, the compiling result and the testing result by using a preset crawler script; and taking the static scanning result, the compiling result, the testing result and the vulnerability information as continuous integration state tracking results of the storage software, and outputting the continuous integration state tracking results to a log file.
By applying the technical scheme provided by the application, in the development process of the storage software, operations such as scanning, compiling, use case testing and the like are sequentially executed on the execution code of the storage software, corresponding static scanning results, compiling results and testing results are obtained, finally, vulnerability scanning of various results is realized by using the preset crawler script, corresponding vulnerability information is obtained, and the static scanning results, the compiling results, the testing results and the vulnerability information are output to the log file as continuous integration state tracking results of the storage software, so that the continuous integration state of the storage software in the development process is tracked in real time, and development efficiency and version quality of the storage software are improved.
In one embodiment of the application, in the process of performing static scanning on the execution code of the storage software based on the tracking command, only the update code of the current tracking task compared with the previous tracking task can be pulled to serve as the execution code to be scanned, and then the static scanning of the execution code is realized by utilizing the git command, so that a final static scanning result is obtained.
The continuously integrated state tracking device for storing software, the electronic device and the computer readable storage medium provided by the application have the technical effects as well, and the application is not repeated here.
Drawings
In order to more clearly illustrate the prior art and the technical solutions in the embodiments of the present application, the following will briefly describe the drawings that need to be used in the description of the prior art and the embodiments of the present application. Of course, the following figures related to the embodiments of the present application are only some of the embodiments of the present application, and it is obvious to those skilled in the art that other figures can be obtained from the provided figures without any inventive effort, and the obtained other figures also belong to the protection scope of the present application.
FIG. 1 is a schematic flow chart of a method for tracking continuously integrated state of stored software;
FIG. 2 is a flow chart of another method for continuously integrating state tracking of stored software provided in the present application;
FIG. 3 is a schematic diagram of a continuously integrated state tracking device for storing software according to the present application;
fig. 4 is a schematic structural diagram of an electronic device provided in the present application.
Detailed Description
The core of the application is to provide a continuous integrated state tracking method of the storage software, which can track the continuous integrated state of the storage software in the development process in real time, thereby effectively improving the development efficiency and version quality of the storage software; another core of the present application is to provide a continuously integrated state tracking device for storing software, an electronic device and a computer readable storage medium, which all have the above beneficial effects.
In order to more clearly and completely describe the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. It will be apparent that the described embodiments are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The embodiment of the application provides a continuous integrated state tracking method for storage software.
Referring to fig. 1, fig. 1 is a flowchart of a continuous integrated state tracking method of storage software provided in the present application, where the continuous integrated state tracking method of storage software is applied to a jenkins tool (an open-source continuous integration tool that provides a friendly operation interface) deployed on a centos (Community Enterprise Operating System ) system, and may include the following S101 to S105.
S101: performing static scanning on the execution code of the storage software according to the tracking command to obtain a static scanning result; the static scan results include project update codes and personal yield codes.
Firstly, it should be noted that the continuous integration state tracking of the storage software provided by the embodiments of the present application is applied to the jenkins tool deployed on the centos system, specifically, the centos system may be deployed in advance on the device where the storage software in the development state is deployed, and then the jenkins tool is deployed in the centos system, so that the continuous integration state tracking of the storage software is implemented by using the jenkins tool. The jenkins tool is an open-source continuous integrated tool for providing friendly operation interfaces.
It will be appreciated that during the software development process, the software delivery pipeline generates release versions from source code in a quick, automated, and repeatable manner. The overall design of how this is done is called "continuous delivery"; the process of starting an assembly line is called "continuous integration"; the process of ensuring quality is called "persistence test"; the process of providing the end product to the user is referred to as "continuous deployment".
Further, this step is intended to enable static scanning of the execution code in the stored software. Specifically, the tracking operation for the continuous integrated state of the storage software responds to the tracking command, and the acquisition mode of the tracking command is not unique, and can be the initiative initiation of a technician based on the user terminal or the automatic response based on the preset triggering condition, which is not limited in the embodiment of the application; further, when a tracking command is received, a pre-deployed static scanning tool can be started according to the tracking command, and meanwhile, an execution code in tracked storage software is acquired, so that the static scanning tool can be used for executing static scanning operation on the execution code to obtain a corresponding static scanning result, and static scanning of the execution code in the storage software is realized.
The static scan results may include, among other things, project update codes, i.e., code amount data of the project dimension (the entire stored software), and personal yield codes, i.e., code amount data of the personal dimension (each of the technicians involved in the development of the stored software).
It should be noted that, the tool type of the pre-deployed static scanning tool does not affect implementation of the technical solution, and the technical solution is set by a technician according to actual situations, which is not limited in the embodiment of the present application. In one possible implementation manner, the static scanning of the execution code of the storage software according to the tracking command to obtain a static scanning result may include: and starting a coverage tool or a Fortify tool or a Pc-link tool according to the tracking command, and performing static scanning on an execution code of the storage software to obtain a static scanning result.
The security (code static security scanning tool) is a quick, accurate and highly extensible static analysis solution, and can help development and security teams solve security and quality defects in the early stage of a software development life cycle, track and manage risks of the whole application combination, and ensure to meet security and coding standards. Fortify (code audit tool) is a static, white-box software source code security test tool that passes through five main analysis engines built in: the method comprises the steps of carrying out static analysis on source codes of application software such as data flow, semantics, structure, control flow, configuration flow and the like, and carrying out comprehensive matching and searching with a software security vulnerability rule set specific to the application software in the analysis process, so that security vulnerabilities existing in the source codes are scanned out and are given a sorting report. PC-Lint (code static inspection tool) is a static code inspection tool developed by GIMPEL company for C/C++, and static code inspection is performed before the code is not run and after the code is compiled.
S102: constructing a compiling project for the static scanning result by using a preset compiling tool to obtain a compiling result; the compiling result comprises the compiling result of each functional module and the whole compiling result.
This step aims at realizing the compiling engineering construction for the static scanning result, and the process can be realized based on a pre-deployed compiling tool (namely the preset compiling tool). Specifically, after static scanning obtains a static scanning result of an execution code of the storage software, a preset compiling tool in the centos system can be started, and the preset compiling tool is utilized to build a compiling project on the static scanning result, so that a corresponding compiling result is obtained, and accordingly, the building and processing of the compiling project are completed. Wherein the compiling result may include a compiling result of each functional module (a compiling result of each functional module in the storage software) and an overall compiling result (a compiling result of the storage software as a whole.)
The static scanning of the execution code and the construction of the compiling engineering can be set as an upstream engineering and a downstream engineering, wherein the static scanning of the execution code is the upstream engineering, the construction of the compiling engineering is the downstream engineering, and the construction of the downstream compiling engineering can be started after the construction of the upstream compiling engineering is finished.
It should be noted that, the tool type of the preset compiling tool does not affect implementation of the technical scheme, and the technical scheme is set by a technician according to actual conditions, which is not limited in the embodiment of the present application. In one possible implementation manner, the constructing a compiling project for the static scan result by using a preset compiling tool to obtain a compiling result may include: and constructing compiling engineering for the static scanning result by using an ant tool or a maven tool or a makefile tool or a make tool to obtain a compiling result.
Wherein, the ant tool and the maven tool are both compiling tools for java language, the makefile tool is a compiling tool for C language, the make tool is used for realizing gcc compiling, the gcc is a compiling tool for C/C++ language, and the make is an incremental (compiling) batch processing tool.
S103: when the compiling engineering is successfully constructed, deploying and executing each test case by using a preset compiling tool to obtain the test result of each test case.
The method aims at realizing test case execution after the construction of the compiling engineering is successful, thereby realizing various functional tests. Specifically, when a message that the compiling engineering is successfully constructed is obtained, a pre-deployed compiling tool (namely the preset compiling tool) in the centos system can be started, and the preset compiling tool is utilized to compile, deploy and execute each test case, so that test results of each test case are obtained after the execution is completed, and therefore the execution of the test cases is realized.
It should be noted that, the tool type of the preset writing tool does not affect implementation of the technical scheme, and the technical scheme is set by a technician according to actual conditions, which is not limited in the embodiment of the present application. In one possible implementation manner, the deploying and executing each test case by using the preset writing tool to obtain a test result of each test case may include: and deploying and executing each test case by using a pytest tool to obtain a test result of each test case.
Among them, pytest is a fully functional Python test tool, which helps to write better programs, similar to Python's own unittest framework, but pytest is more compact and efficient to use and compatible with unittest framework.
S104: and obtaining vulnerability information of the static scanning result, the compiling result and the testing result by using a preset crawler script.
The step aims at achieving the acquisition of the vulnerability information, wherein the vulnerability information comprises vulnerability information of a static scanning result, vulnerability information of a compiling result and vulnerability information of a testing result. Specifically, after the static scanning result, the compiling result and the testing result are obtained respectively, the three types of results can be respectively crawled and analyzed by utilizing the pre-deployed crawler script (namely the preset crawler script), so as to obtain the corresponding vulnerability information, and therefore, the analysis and the acquisition of the vulnerability information are realized.
The preset crawler script is a pre-written crawler script and is deployed in the centos system, and specifically, python can be used for writing the preset crawler script. Furthermore, the preset crawler script may be directly used as the last test case of the test cases in S103, and may be directly executed after the other test cases are completed.
In an embodiment of the present application, the obtaining vulnerability information of the static scan result, the compiling result, and the testing result by using the preset crawler script may include:
crawling preset key rows in the static scanning result by using a preset crawler script to obtain vulnerability information of the static scanning result and file path information corresponding to the vulnerability information;
crawling the compiling results of each functional module by using a preset crawler script, and obtaining failure compiling information corresponding to the compiling results of the functional modules as vulnerability information of the compiling results;
and crawling the test result by using a preset crawler script to obtain the vulnerability information of the test case which fails in the test as the test result.
The embodiment of the application provides a method for achieving vulnerability information acquisition based on a preset crawler script. For the static scanning result, the preset crawler script can be utilized to crawl the preset key row (usually the last row) in the static scanning result to obtain corresponding vulnerability information (including the number of vulnerabilities and the vulnerability content) and the file path of each vulnerability. For the compiling result of the functional module, the compiling result of each functional module can be crawled by using a preset crawler script, when the failure compiling information of the compiling result of a certain functional module is crawled, the failure compiling information of the compiling result of the functional module can be used as the vulnerability information of the compiling result of the functional module, and generally, the last 5 lines of the compiling result of the functional module can be intercepted to be used as the compiling failure reason, namely the vulnerability information. For the test cases, the test results corresponding to the test cases can be directly crawled by using the preset crawler script, the test cases with test results of test failure are used as corresponding vulnerability information, and the method can further comprise the number of the test cases with test failure, the number of the test cases with test success, the total time consumption of the test and the like.
S105: and taking the static scanning result, the compiling result, the testing result and the vulnerability information as continuous integrated state tracking results of the storage software, and outputting the continuous integrated state tracking results to the log file.
This step aims at realizing the determination of the tracking result of the continuous integration state of the stored software and the output thereof. Specifically, after static scanning of the execution code, construction of compiling engineering, execution of the test case and vulnerability scanning of the execution results of the three, the corresponding results (including the static scanning result, compiling result, testing result and vulnerability information of the three) can be synthesized to be used as a tracking result of the continuous integration state of the storage software, and finally output to a pre-deployed log file.
Therefore, in the method for tracking the continuous integrated state of the storage software, which is provided by the embodiment of the application, operations such as scanning, compiling, use case testing and the like are sequentially executed on the execution code of the storage software in the development process of the storage software, so that corresponding static scanning results, compiling results and testing results are obtained, finally, vulnerability scanning of various results is realized by using a preset crawler script, corresponding vulnerability information is obtained, and the static scanning results, compiling results, testing results and vulnerability information are output to a log file as the continuous integrated state tracking result of the storage software, so that the continuous integrated state of the storage software in the development process is tracked in real time, and development efficiency and version quality of the storage software are improved.
Based on the above embodiments:
in one embodiment of the present application, the performing static scanning on the execution code of the storage software according to the tracking command to obtain a static scanning result includes:
pulling an execution code in the storage software according to the tracking command, wherein the execution code is an update code of the current tracking task compared with the last tracking task;
and scanning and analyzing the execution code by using the git command to obtain a static scanning result.
The embodiment of the application provides an implementation mode for carrying out static scanning on an execution code of stored software to obtain a static scanning result. Specifically, after receiving the tracking command, the execution code, specifically the update code of the current tracking task compared with the previous tracking task, can be pulled in the storage software according to the tracking command, and not all the codes in the storage software; further, static scanning analysis is carried out on the execution code by utilizing the git command, and a corresponding static scanning result can be obtained. The operating principle of the git command is that a snapshot of a project is created and saved, and compared with a subsequent snapshot, and the snapshot is a version control technology from a developer to a designer.
The current tracking task is a continuous integrated state tracking task of the storage software responding to the current tracking command, and the last tracking task is a continuous integrated state tracking task of the storage software responding to the last tracking command. It may be understood that the tracking command may be an automatic triggering command based on a preset time interval, and the corresponding code is updated between two times of initiation time of the tracking command, where the executing code of the storage software is statically scanned according to the tracking command, and before obtaining the static scanning result, the method may further include: the tracking command is automatically triggered at preset time intervals. The specific value of the preset time interval does not affect implementation of the technical scheme, and the technical scheme is set by a technician according to actual conditions, for example, the preset time interval can be 24 hours (1 day), and the embodiment of the application is not limited to the specific value.
It can be seen that, in the embodiment of the application, in the process of performing static scanning on the execution code of the storage software based on the tracking command, only the update code of the current tracking task compared with the previous tracking task can be pulled to be used as the execution code to be scanned, and then the static scanning of the execution code is realized by using the git command, so that a final static scanning result is obtained.
In one embodiment of the present application, the pulling the execution code in the storage software according to the tracking command, and performing scan analysis on the execution code by using the git command to obtain a static scan result includes:
comparing and analyzing the code of the current tracking task and the code of the last tracking task by utilizing the gitdiff command to obtain an analysis result;
extracting preset key fields of the analysis result to obtain an execution code, and taking the execution code as an item updating code;
acquiring a code submission record of the storage software between a current tracking task and a last tracking task by utilizing a git log command;
performing duplication removal processing on the code submission record through a sort-u command to obtain a duplicated code submission record;
traversing in the code submitting record after the duplication removal to obtain submitting user information;
acquiring a submitting record of a submitting user corresponding to each submitting user information between a current tracking task and a last tracking task by utilizing a git log command;
and obtaining the yield codes of the submitting users in each submitting record by utilizing the awk command, and taking the yield codes of the submitting users as personal yield codes.
The embodiment of the application provides a method for realizing project update codes and personal output codes, namely, based on a gitdiff command, a gitlog command, a sort-u command, an awk command and the like in a git command, the gitdiff command is used for realizing the acquisition of the project update codes, and the rest is used for realizing the acquisition of the personal output codes.
For the project update code, the statistical scale of the project code can be calculated by comparing two separate completions (completions a and completions X, i.e., the code of the current trace task and the code of the previous trace task), in which process, a gitdiff command can be used to output a code value in the format of "X files changed, X inserts (+), X inserts (-)", where the "inserts" field is the code value of the new+modification, i.e., the project update code.
For the personal yield code, firstly, acquiring all commit records on project branches by using a git log command, and transmitting the result to a sort-u command through a pipeline to ensure that only unique commit records are acquired; secondly, the author names (names) of submitted records (submitted user information) are read one by one using a while loop; then, for each author, get its commit record on the SpruceV1 branch using the git log command and pass the result through the pipeline to the awk command; further, in the awk command, the number of lines added (add), the number of lines deleted (subs) and the total number of lines (loc) by each author may be calculated, and finally, the add item is focused to obtain the yield code of each person.
In one embodiment of the present application, the method for continuously integrating state tracking of stored software may further include:
When the construction of the compiling engineering fails, utilizing a preset crawler script to acquire vulnerability information of a static scanning result;
and taking the static scanning result, vulnerability information of the static scanning result and the compiling result as continuous integrated state tracking results of the storage software, and outputting the continuous integrated state tracking results to the log file.
Specifically, when the compiling engineering fails to build, the publication cannot be normally built for automatic testing, so that the continuous integration flow can be directly terminated, at this time, the vulnerability information of the static scanning result is obtained by directly using the preset crawler overtime, the vulnerability information of the static scanning result and the compiling result are taken as continuous integration state tracking results of the storage software together, and finally the continuous integration state tracking results are output to the log file.
Based on the above embodiments, another method for continuously tracking the integrated state of the stored software is provided in the embodiments of the present application.
Referring to fig. 2, fig. 2 is a flow chart of another continuous integrated state tracking method for storing software provided in the present application, and the implementation flow is as follows:
1. continuously integrated timing construction configuration:
1. tool selection for continuous integration:
in the continuous integration whole flow, reasonable tool selection is needed to reasonably and efficiently complete each step of each functional module, and the tool selection is carried out on main steps of continuous integration, specifically as follows:
(1) Code checking and security scanning: a Coverity, fortify and other weight-class code security scanning tool can be selected, and pc-lin and the like can also be selected;
(2) Building and compiling: for java, selecting ant, maven and other compiling tools; for the c language, makefile can be selectively written; gcc compilation using make;
(3) Automated testing: pytest can be selected for automation script (test cases) writing, and fabric3 (used for realizing automation deployment) is used as a communication mode among storage, a server host and the like;
(4) bug (vulnerability) record: a mantis (defect management tool) or the like may be used as the bug record.
2. Tool timing initiation trigger configuration:
(1) Jenkins and centos systems are used as a main trigger platform for continuously integrating functional modules: installing a centos system on a device where the storage software is located, and installing a jenkins tool on the centos system;
(2) Setting a code scanning timing task: jenkins sets a timing task in the early morning every day, pulls the latest code, and performs static code checking scanning on the code;
(3) Calculating the latest code pulled by using a git command, and analyzing code amount data (project update code and personal yield code) of the personal dimension and the project dimension:
(1) Project update code statistical scale calculation:
git diffcommitA commitX- -stat// comparison of two separate completions, output as X files changed, X inserts (+), X reductions (-). The output uses the fields of inserts, i.e. the code value of add + modify.
(2) Personal yield code calculation:
the git log entry branch name commit a … commit-format= "% aN" |sort-u|while read name; do echo-en "$name\t"; the git log SpruceV1- -author= "$name" commit A … commit- -pretty=tformat: -numstat|awk' { add+ = $1; subs+ = $2; loc+ = $1- $2}END{printf"added lines%s, moved lines%s, total lines%s\n", add, subs, loc' -; done looks at add items.
(4) Triggering and constructing: setting up an upstream project and a downstream project, taking static inspection scanning as the upstream project, and starting the downstream construction compiling project after finishing.
(5) Triggering and testing: after the construction and compiling engineering is finished, an automatic test interface can be triggered through a command line to automatically deploy and execute test cases.
(6) Triggering a script for acquiring the bug: and when the automatic test is executed, executing the script to acquire the integral bug of the current day and the solution condition of the integral bug.
2. And (3) obtaining execution results of all steps in the continuous integration process:
in the whole continuous integration whole flow, the execution results of all steps need to be saved, and a total log file, such as ci.log, can be created in advance:
1. static inspection scan results: and (3) acquiring the last key row of the static inspection scanning result, wherein the quantity of the high, medium and low-level vulnerabilities is input into the ci.log, and simultaneously acquiring file paths of the vulnerabilities at all levels and inputting the file paths into the ci.log.
2. Code statistics: the project and individual daily newly developed code amount calculations are saved and entered into ci.log.
3. Construction results:
(1) Constructing a whole by adopting jenkins pipeline to compile each dependency module and packaging the whole, defining FAILD_STAGE variables as records of FAILED STAGE, outputting respective compiling processes to log files named by module NAMEs when each dependency module is compiled, for example, compiling an os module of a storage system, outputting a STAGE (os) compiling process to an os.log, defining FAILED_STAGE=env.STAGE_NAME in each STAGE opening, and when certain STAGE compiling fails, if the os compiling fails, acquiring FAILED STAGE as os in the jenkins file of the pipeline by using FAILE_STAGE, checking the os.log, intercepting the last 5 rows as failure reasons and printing the last 5 rows into ci.log;
(2) If the construction result fails, the version cannot be normally constructed for the automatic test, the continuous integration flow can be directly stopped, and if the construction is successful, the automatic test is triggered to automatically transmit the package, deploy, execute the use case and the like.
4. Test results: after the test flow taking pytest as a basic framework is triggered after construction, the total number of test cases, the number of success and failure, the total test time consumption and the like are obtained from a test report and are input into ci.log.
5. bug solving: the last use case in the test process triggers a crawler script, and the total number of project bug, the solved number, the current responsible person of bug and the like are obtained from the bug system and output to the ci.log.
3. The log content is spliced into html:
1. in the process of continuously integrating the input of each step to ci.log, the marks of [ step name ] can be input at the beginning to perform step differentiation.
2. Reading, analyzing and splicing the steps into a table, and taking the following contents as a table main body:
(1) The execution content is as follows: static checking scanned code branches and scanning projects, code branches and commit information of each module during construction, all construction parameters, test task names and equipment;
(2) Execution result: constructing successful, failed and failed stage, consuming time for constructing, statically checking the number of high, medium and low loopholes, testing the total number of task use cases, the number of successful and failed use cases and consuming time for executing the use cases;
(3) Failure cause: constructing detailed failure information of failed stage, statically checking file paths of all vulnerabilities and use case paths of test failures;
(4) Code amount data for project and personal dimensions.
3. And (3) displaying the whole page: considering that many data have lengthy problems, such as 500 test cases failing 300, but not 300, a partial display of the failure cause column may be performed, i.e., a partial failure cause data may be displayed in the archived display range, and an ellipsis added at the end, and clicking on the ellipsis may display the complete information.
4. E, mail sending:
1. a send mail function is composed using a python mail sending module or jenkins;
2. when the continuous integrated construction link fails, the mail is directly triggered to be sent, the continuous execution of the steps such as testing is not carried out, and the mail content only displays static checking and construction information; when the construction is successful, the full content is sent, and html mails spliced by the log content are sent to all relevant module interface persons, responsible persons, software managers and the like during the sending.
Therefore, in the method for tracking the continuous integrated state of the storage software, which is provided by the embodiment of the application, operations such as scanning, compiling, use case testing and the like are sequentially executed on the execution code of the storage software in the development process of the storage software, so that corresponding static scanning results, compiling results and testing results are obtained, finally, vulnerability scanning of various results is realized by using a preset crawler script, corresponding vulnerability information is obtained, and the static scanning results, compiling results, testing results and vulnerability information are output to a log file as the continuous integrated state tracking result of the storage software, so that the continuous integrated state of the storage software in the development process is tracked in real time, and development efficiency and version quality of the storage software are improved.
The embodiment of the application provides a continuously integrated state tracking device for storing software.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a continuously integrated state tracking device for storing software provided in the present application, where the continuously integrated state tracking device for storing software is applied to a jenkins tool deployed on a centos system, and may include:
the scanning module 1 is used for carrying out static scanning on the execution codes of the storage software according to the tracking command to obtain a static scanning result; the static scanning result comprises an item updating code and a personal yield code;
the compiling module 2 is used for constructing compiling engineering for the static scanning result by utilizing a preset compiling tool to obtain a compiling result; the compiling result comprises a single module compiling result and an overall compiling result;
the execution module 3 is used for deploying and executing each test case by using a preset writing tool when the compiling engineering construction is successful, so as to obtain the test result of each test case;
the crawling module 4 is used for acquiring vulnerability information of a static scanning result, a compiling result and a testing result by utilizing a preset crawler script;
and the output module 5 is used for taking the static scanning result, the compiling result, the testing result and the vulnerability information as continuous integrated state tracking results of the storage software and outputting the continuous integrated state tracking results to the log file.
Therefore, in the continuous integrated state tracking device for the storage software provided by the embodiment of the application, operations such as scanning, compiling, use case testing and the like are sequentially executed on the execution code of the storage software in the development process of the storage software, so that corresponding static scanning results, compiling results and testing results are obtained, finally, vulnerability scanning of various results is realized by using the preset crawler script, corresponding vulnerability information is obtained, and the static scanning results, compiling results, testing results and vulnerability information are output to the log file as the continuous integrated state tracking results of the storage software, so that the continuous integrated state of the storage software in the development process is tracked in real time, and development efficiency and version quality of the storage software are improved.
In one embodiment of the present application, the above-mentioned scanning module 1 may be specifically configured to start a coverage tool, a Fortify tool, or a Pc-line tool according to a tracking command, and perform static scanning on an execution code of the stored software to obtain a static scanning result.
In one embodiment of the present application, the scanning module 1 may be specifically configured to pull an execution code in the storage software according to the tracking command, where the execution code is an update code of a current tracking task compared to a previous tracking task; and scanning and analyzing the execution code by using the git command to obtain a static scanning result.
In one embodiment of the present application, the compiling module 2 may be specifically configured to construct a compiling project for the static scan result by using an ant tool or a maven tool or a makefile tool or a make tool, so as to obtain a compiling result.
In one embodiment of the present application, the execution module 3 may be specifically configured to deploy and execute each test case by using a pytest tool, so as to obtain a test result of each test case.
In one embodiment of the present application, the crawling module 4 may be specifically configured to crawl a preset key row in the static scan result by using a preset crawler script, to obtain vulnerability information of the static scan result and file path information corresponding to the vulnerability information; crawling the compiling results of each functional module by using a preset crawler script, and obtaining failure compiling information corresponding to the compiling results of the functional modules as vulnerability information of the compiling results; and crawling the test result by using a preset crawler script to obtain the vulnerability information of the test case which fails in the test as the test result.
In an embodiment of the present application, the continuously integrated state tracking device of the storage software may further include a first output module, configured to obtain vulnerability information of the static scan result by using a preset crawler script when the compiling engineering construction fails; and taking the static scanning result, vulnerability information of the static scanning result and the compiling result as continuous integrated state tracking results of the storage software, and outputting the continuous integrated state tracking results to the log file.
For the description of the apparatus provided in the embodiment of the present application, reference is made to the above method embodiment, and the description is omitted herein.
The embodiment of the application provides electronic equipment.
Referring to fig. 4, fig. 4 is a schematic structural diagram of an electronic device provided in the present application, where the electronic device may include:
a memory for storing a computer program;
a processor for implementing the steps of any of the above-described continuous integrated state tracking methods of stored software when executing a computer program.
As shown in fig. 4, which is a schematic diagram of a composition structure of an electronic device, the electronic device may include: a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 all complete communication with each other through a communication bus 13.
In the present embodiment, the processor 10 may be a central processing unit (Central Processing Unit, CPU), an asic, a dsp, a field programmable gate array, or other programmable logic device, etc.
The processor 10 may call a program stored in the memory 11, and in particular, the processor 10 may perform operations in an embodiment of a continuously integrated state tracking method of stored software.
The memory 11 is used for storing one or more programs, and the programs may include program codes, where the program codes include computer operation instructions, and in this embodiment, at least the programs for implementing the following functions are stored in the memory 11:
performing static scanning on the execution code of the storage software according to the tracking command to obtain a static scanning result; the static scanning result comprises an item updating code and a personal yield code;
constructing a compiling project for the static scanning result by using a preset compiling tool to obtain a compiling result; the compiling result comprises the compiling result of each functional module and the whole compiling result;
when the compiling engineering is successfully constructed, deploying and executing each test case by using a preset compiling tool to obtain a test result of each test case;
obtaining vulnerability information of a static scanning result, a compiling result and a testing result by using a preset crawler script;
and taking the static scanning result, the compiling result, the testing result and the vulnerability information as continuous integrated state tracking results of the storage software, and outputting the continuous integrated state tracking results to the log file.
In one possible implementation, the memory 11 may include a storage program area and a storage data area, where the storage program area may store an operating system, and at least one application program required for functions, etc.; the storage data area may store data created during use.
In addition, the memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid-state storage device.
The communication interface 12 may be an interface of a communication module for interfacing with other devices or systems.
Of course, it should be noted that the structure shown in fig. 4 is not limited to the electronic device in the embodiment of the present application, and the electronic device may include more or fewer components than those shown in fig. 4 or may combine some components in practical applications.
Embodiments of the present application provide a computer-readable storage medium.
The computer readable storage medium provided in the embodiments of the present application stores a computer program, where the computer program when executed by a processor may implement any one of the steps of the continuous integrated state tracking method for storage software.
The computer readable storage medium may include: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
For the introduction of the computer readable storage medium provided in the embodiments of the present application, reference is made to the above method embodiments, and the description is omitted herein.
In the description, each embodiment is described in a progressive manner, and each embodiment is mainly described by the differences from other embodiments, so that the same similar parts among the embodiments are mutually referred. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The technical scheme provided by the application is described in detail. Specific examples are set forth herein to illustrate the principles and embodiments of the present application, and the description of the examples above is only intended to assist in understanding the methods of the present application and their core ideas. It should be noted that it would be obvious to those skilled in the art that various improvements and modifications can be made to the present application without departing from the principles of the present application, and such improvements and modifications fall within the scope of the present application.
Claims (10)
1. A method of continuous integrated state tracking of stored software, applied to jenkins tools deployed on a centos system, comprising:
performing static scanning on the execution code of the storage software according to the tracking command to obtain a static scanning result; the static scanning result comprises an item updating code and a personal yield code;
constructing a compiling project for the static scanning result by using a preset compiling tool to obtain a compiling result; the compiling result comprises a compiling result of each functional module and an overall compiling result;
when the compiling engineering is successfully constructed, deploying and executing each test case by using a preset compiling tool to obtain a test result of each test case;
Obtaining vulnerability information of the static scanning result, the compiling result and the testing result by using a preset crawler script;
and taking the static scanning result, the compiling result, the testing result and the vulnerability information as continuous integration state tracking results of the storage software, and outputting the continuous integration state tracking results to a log file.
2. The method for tracking the continuous integrated state of the storage software according to claim 1, wherein the performing static scanning on the execution code of the storage software according to the tracking command to obtain a static scanning result comprises:
and starting a coverage tool or a Fortify tool or a Pc-link tool according to the tracking command, and performing static scanning on an execution code of the storage software to obtain a static scanning result.
3. The method for tracking the continuous integrated state of the storage software according to claim 1, wherein the performing static scanning on the execution code of the storage software according to the tracking command to obtain a static scanning result comprises:
pulling the execution code in the storage software according to the tracking command, wherein the execution code is the update code of the current tracking task compared with the last tracking task;
And scanning and analyzing the execution code by using a git command to obtain the static scanning result.
4. The continuous integrated state tracking method of stored software according to claim 1, wherein the constructing a compiling project for the static scan result by using a preset compiling tool to obtain a compiling result includes:
and constructing the compiling engineering for the static scanning result by using an ant tool or a maven tool or a makefile tool or a make tool to obtain the compiling result.
5. The method for tracking the continuous integrated state of the storage software according to claim 1, wherein the deploying and executing each test case by using a preset writing tool to obtain a test result of each test case comprises:
and deploying and executing each test case by using a pytest tool to obtain a test result of each test case.
6. The method for continuously tracking the integrated state of the storage software according to claim 1, wherein the obtaining vulnerability information of the static scan result, the compiling result, and the testing result by using a preset crawler script includes:
crawling preset key rows in the static scanning result by using the preset crawler script to obtain vulnerability information of the static scanning result and file path information corresponding to the vulnerability information;
Crawling the compiling results of each functional module by using the preset crawler script, and obtaining failure compiling information corresponding to the compiling results of the functional modules as vulnerability information of the compiling results;
and crawling the test result by using the preset crawler script to obtain a test case with failed test as vulnerability information of the test result.
7. The method of continuous integrated state tracking of stored software according to claim 1, further comprising:
when the compiling engineering construction fails, utilizing the preset crawler script to acquire vulnerability information of the static scanning result;
and taking the static scanning result, vulnerability information of the static scanning result and the compiling result as continuous integrated state tracking results of the storage software, and outputting the continuous integrated state tracking results to the log file.
8. A continuously integrated state tracking device storing software, for use with a jenkins tool deployed on a centos system, comprising:
the scanning module is used for carrying out static scanning on the execution codes of the storage software according to the tracking command to obtain a static scanning result; the static scanning result comprises an item updating code and a personal yield code;
The compiling module is used for constructing compiling engineering for the static scanning result by utilizing a preset compiling tool to obtain a compiling result; the compiling result comprises a single-module compiling result and an overall compiling result;
the execution module is used for deploying and executing each test case by using a preset writing tool when the compiling engineering is successfully constructed, and obtaining the test result of each test case;
the crawling module is used for acquiring vulnerability information of the static scanning result, the compiling result and the testing result by utilizing a preset crawler script;
and the output module is used for taking the static scanning result, the compiling result, the testing result and the vulnerability information as continuous integrated state tracking results of the storage software and outputting the continuous integrated state tracking results to a log file.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the continuous integrated state tracking method of stored software according to any one of claims 1 to 7 when executing said computer program.
10. A computer readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the continuous integrated state tracking method of storing software according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311778124.8A CN117707590A (en) | 2023-12-22 | 2023-12-22 | Continuous integrated state tracking method, device, equipment and medium for storage software |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311778124.8A CN117707590A (en) | 2023-12-22 | 2023-12-22 | Continuous integrated state tracking method, device, equipment and medium for storage software |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117707590A true CN117707590A (en) | 2024-03-15 |
Family
ID=90145906
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311778124.8A Pending CN117707590A (en) | 2023-12-22 | 2023-12-22 | Continuous integrated state tracking method, device, equipment and medium for storage software |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117707590A (en) |
-
2023
- 2023-12-22 CN CN202311778124.8A patent/CN117707590A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3769223B1 (en) | Unified test automation system | |
| Fagan | Design and code inspections to reduce errors in program development | |
| CN101046767B (en) | Method and system for automated testing of a graphic-based programming tool | |
| US7596778B2 (en) | Method and system for automatic error prevention for computer software | |
| US7490319B2 (en) | Testing tool comprising an automated multidimensional traceability matrix for implementing and validating complex software systems | |
| US20030046029A1 (en) | Method for merging white box and black box testing | |
| Tsai et al. | Scenario-based functional regression testing | |
| CN108845940B (en) | Enterprise-level information system automatic function testing method and system | |
| US8549483B1 (en) | Engine for scalable software testing | |
| US20080109790A1 (en) | Determining causes of software regressions based on regression and delta information | |
| CN106909510A (en) | A kind of method and server for obtaining test case | |
| Nguyen et al. | A goal-oriented software testing methodology | |
| Valenzuela-Toledo et al. | Evolution of github action workflows | |
| CN112131116B (en) | Automatic regression testing method for embedded software | |
| CN103186463B (en) | Determine the method and system of the test specification of software | |
| Ganesan et al. | Verifying architectural design rules of the flight software product line | |
| US20190236223A1 (en) | Identification of changes in functional behavior and runtime behavior of a system during maintenance cycles | |
| Hue et al. | USLTG: test case automatic generation by transforming use cases | |
| CN113127280A (en) | API interface automatic input method and system | |
| Kabadi et al. | The Future Can’t Help Fix The Past: Assessing Program Repair In The Wild | |
| CN117707590A (en) | Continuous integrated state tracking method, device, equipment and medium for storage software | |
| Lübke | Extracting and conserving production data as test cases in executable business process architectures | |
| Rwemalika et al. | Ukwikora: continuous inspection for keyword-driven testing | |
| Talby et al. | A process-complete automatic acceptance testing framework | |
| Braunisch et al. | Maturity Evaluation of SDKs for I4. 0 Digital Twins |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |