CISQ and Structural Quality

The Consortium for IT Software Quality (CISQ) recently made an announcement related to software quality. CISQ is a relatively young organization, founded in 2009 by Carnegie Mellon’s Software Engineering Institute (the organization that initiated the CMM process standards) and a slightly lesser-known organization, Object Management Group (OMG), an IT standard technology non-profit organization that started its existence focusing on object-oriented systems.

CISQ wants to set up standards for defining - and measuring - the quality of a software product or custom application from a structural perspective. To do this, the organization has focused on several (but not all) criteria mentioned by the International Organization for Standardization (ISO) i.e. reliability, security, performance, efficiency and maintainability, and has defined those criteria. This is not as abstract as it might sound: one of the goals of CISQ is to turn those criteria into items (e.g. tokens, objects or control structures) that can be measured automatically. In short, CISQ has defined the standards to help measure violations of good coding practices e.g. SQL injections, circular dependencies, loop operations, or dead code.

Key in this effort, beyond the definition of practical standards, is the drive to develop automated measurement - this will help drive adoption. CISQ has a track record in driving standards towards practical usage: the organization worked alongside partners on the definition of function points (for development projects) and driving automated counting. While most people will be familiar with function points, many of us may not have realized that the definition of function points was fairly loose and two IT analysts trying to capture the number of function points in a given application would get different function point number estimates. So CISQ worked on defining what function points are: this should pave the way for automated counting of function points in the near future. This is all new of course as CISQ only released the details of function points in 2015.

Software Testing and Functional Quality

Now looking to the other side of software quality, at functional (and also non-functional) quality, which largely been the focus area of software testing.

Software testing has largely expanded from its pass/fail mission towards Quality Assurance (QA). QA is probably an overused term in the software testing industry but it has its benefits: it relates to several specific activities such as test process improvement (from testing requirement to test execution) and test project management - other activities such as testing tool consulting, or advisory services to centralizing testing service are less relevant. Test process improvement and test project management have helped in expanding the focus of testing to the full testing life cycle.

Yet, QA aims at improving software testing (making it more efficient and more professional) not at designing and developing good quality software. Internal testing teams and software testing services have done an immense work in terms of further automating testing (a labor-intensive activity), and adapting to Agile, DevOps, and Continuous Integration. All are game-changers: broadly speaking, they promote an iterative and automated approach to software development, testing and release into production. As such they go beyond the testing life cycle to the full SLDC. Obviously, this is a significant step and the transition is complex to set up in terms of organization, tools, best practices – and culture.

Structural Quality and Functional Quality: Complementary but Non-Integrated

So with the practice structural quality standards defined by CISQ and the very broad and increasingly automated services delivered by the software testing services industry, we have two complementary approaches to software quality. This is good news. But we also have significant challenges in front of us: CISQ is still years away from having its standards being adopted by clients and the testing ecosystems (testing tool ISVs and testing service vendors). In the short term, we think there potentially is a fantastic opportunity for software testing vendors in terms of consulting services, also for tech start-ups. Though, none of the software testing services units we have talked seems to be involved in this at this point. We would like see this change.

NelsonHall is currently working on its next release of its industry-leading Software Testing Analysis and Forecast. For more information, please contact Guy Saunders at [email protected].