Implementing parallel software is hard. On top of all the kinds of bugs that may be present in sequential code, parallel code can also suffer from race conditions. A race condition exists when several threads access the same data at the same time and at least one of them writes to it.
Opening the door to new bugs is bad enough, but to make matters worse, the bugs related to race conditions happen to be very hard to debug. A parallel code might run correctly 99% of the times and fail just the remaining 1%. This is due to the fact that the order in which the different code paths are run concurrently or in parallel can differ between executions. The bug may even arise only when the number of simultaneous code paths is increased above a certain threshold. This makes the code unpredictable. Many times, developers have no clue how to reproduce the bug, which makes fixing it extremely difficult and time-consuming.
The cost of bugs in parallel software
According to Capers Jones, the cost of finding and fixing bugs/defects while testing software is 4x-40x more expensive than in the coding stage and it raises up to 640x after releasing the software to customers. Up to 85% of the bugs are introduced while coding computer programs, while most of them are detected and fixed later while debugging the code during testing. In addition, according to the University of Cambridge, software developers spend 30% of their time in debugging and we estimate that another 50% is spent in coding.
In order to reduce software development costs and speed up the software development process, it is key to shift bug detection to an earlier development phase and to automate it during coding and testing. This is particularly important for the development and maintenance of parallel software, which is far more complex than sequential software. Bugs related to parallelism are more likely to go undiscovered. They may manifest in production only in some very specific or non-deterministic scenarios, causing an enormous cost.
Quality assurance and compliance for parallel software
In certain industries such as automotive and cybersecurity, automation and speedup of the coding and testing is often accomplished by using static code analysis tools to ensure compliance with coding standards (e.g., MISRA, CWE, CERT). The rationale is that by ensuring that the code adheres to best practices and recommendations, the resulting software is less likely to contain bugs. Some tools can even detect bugs already present in the code.
Due to the increasing adoption of parallel computing and the ever-increasing complexity of parallel software, it is becoming critical that best practice recommendations for parallel code are also covered by coding standards. Although there is currently no coding standard or formal guidelines for parallelism, some attempts have been made to cover some of the issues in this field by more general initiatives. Such an example is the SEI CERT Coding Standard, whose rule 14 includes 13 recommendations for concurrency.
There are also some early initiatives that are curating errors commonly seen in parallel codes. For instance, the DataRaceBench benchmark suite provides example codes to validate whether analysis tools successfully detect issues related to parallelism and the IPT interactive tool shows how a set of code examples should be parallelized. The clang-tidy C++ linter tool provides a couple of checks for OpenMP code. There are also very focused guides such as Intel’s Guide to vectorization with Intel C++ compilers which contains valuable general information on vectorization.
The interest on this topic is growing, as evidenced by the upcoming Quality Assurance and Coding Standards for Parallel Software session in the Supercomputing 2019 conference.
A catalog of defects and recommendations for parallelism
Aiming at helping fill this void, we are currently working on creating a catalog of defects and recommendations related to parallelism. Defects are software errors which are normally not reported during compilation and that lead to bugs at runtime. Recommendations are software warnings regarding code which is not compliant with best practices and which may or may not lead to bugs at runtime. The umbrella term check serves to refer to both defects and recommendations.
This is a huge endeavor in constant evolution. We are constantly engaging with the community to develop the most relevant checks for parallelism. Thus, the list is growing every day. Detailed up-to-date information on all the defects and recommendations is located at: https://www.appentra.com/knowledge/checks/
State-of-the-art static code analysis for parallelism with Parallelware Analyzer
Parallelware Analyzer includes a next-generation static code analysis tool providing an ever-increasing number of checks for parallelism. Designed with DevOps in mind, it is trivial to integrate it into your Continuous Integration system to help ensure that your code is defect-free and compliant with best practices for parallelism.
The following table summarizes some metrics resulting of running Parallelware Analyzer for 139 benchmarks from well-known public benchmark suites, including the NAS Parallel Benchmarks, CORAL, DSPstone, Patmos, MiBench and TSVC.
|Single lines of code||15.410||2.141.958|
|Execution time (seconds)||3||418|
Parallelware Analyzer needed only 7 minutes to complete the analysis of more than 2M lines of code spread across more than 15K sources files. The average run needed only 3 seconds to complete the analysis of 15K lines of code. For the OpenMP version of the SNU NAS Parallel Benchmarks alone, it analyzed 20.824 lines of code across 106 source files in 5 seconds to find 115 opportunities for parallelization and report 1.579 recommendations.
Take a look at this post presenting a real use case of Paralleware Analyzer to detect and fix defects in OpenMP code.
Join the Early Access Program to try out Paralleware Analyzer!