Review the Concepts in This Chapter That Are Related to Improving the Quality of Software

Abstract

All engineering disciplines have notions of production quality. Along with these notions come mechanisms and all-time practices ensuring that, for a given product, each item of the production has a specified quality. Furthermore, we are used to thinking that the most disquisitional of these quality metrics are absolute. If the product fails to run across these absolute quality metrics, the customer might have legal claims on the producer. Such quality breaches are therefore expected to be relatively rare in most engineering science disciplines.

All engineering science disciplines accept notions of production quality. Along with these notions come mechanisms and best practices ensuring that, for a given production, each detail of the product has a specified quality. Furthermore, nosotros are used to thinking that the about critical of these quality metrics are absolute. If the product fails to meet these absolute quality metrics, the customer might have legal claims on the producer. Such quality breaches are therefore expected to exist relatively rare in most engineering disciplines.

Our expectations of the quality of software are unlike. For software running on standard consumer equipment, we expect a stream of security updates. These updates fix security holes that clearly must accept been in the product that was bought prior to the installation of the update. This situation is non the result of the fail of the software engineering community or the result of a lack of investment in quality assurance by the manufacturers. Instead, it is the issue of inherent challenges in the concept of software evolution. The support of quality through business firm notions and mathematical rigour existing in many other engineering fields has been shown to be difficult to replicate in a scalable way in software engineering (meet Chap. nine).

Still, the quality assurance of software has received a massive corporeality of attending. In this affiliate, we provide an overview of the nearly important developments, aiming to meet if methods have been adult that tin can aid accost our question.

What is Software Quality Direction?

Early in the development of computers, it became apparent that the instructions that guide the performance of machines could be stored as data. This naturally pointed towards the formation of a new notion, software, that encompassed the executable files in a computer system. The flexibility of separation between the physical machine itself and its operating instructions was shown to exist immensely powerful and software shortly developed into an engineering subject field in its own right.

Before long, however, it became evident that the technology of software presented its ain very specific challenges. Unintentional errors in the code appeared to be hard to avert and, when created, equally difficult to detect and correct. Throughout the decades, a great deal of effort has been invested into finding ways to ensure the quality of developed software. These investments have provided insights into some key properties that make software quality balls different from what we observe in other engineering disciplines [14].

Complication :

1 way of measuring the complexity of a product is to count the number of distinct operating modes in which it is supposed to part. A slice of software may be required to work on pinnacle of unlike platforms, work together with different sets of other software running alongside, and be guided past a prepare of parameters and settings that tin be freely chosen by the user. The combinatoric explosion of possible configuration settings for a software packet easily runs into the millions. Traditional applied science artefacts rarely see them run into just the thousands.

Visibility :

Software is invisible. This means that the opportunity to discover mistakes without observing their consequences is limited to very few people. In nigh physical products, many mistakes are impossible but considering they would be obvious to the eye of anyone looking at them.

Manufacturing :

The production phase is very unlike in software engineering, since it just consists of making some files available, either through download or through the standardized process of producing storage media with the files on them. There is no product planning phase in which mistakes can be discovered and the manufacturing is carried out without anyone seeing the product.

Assuring the quality of software is therefore different from assuring the quality of most other products. Because of the general invisibility of software and the fact that its quality is not influenced past the process of manufacturing copies of it, software quality assurance is aimed at the production and maintenance phases. The master elements of software quality assurance are the development process, the quality model, and the implemented quality management scheme. These three notions are described in the upcoming sections.

Software Development Process

The initial waterfall model for software development was described in the early 1970s. Many variations of the model exist but they mostly consisted of a series of stages, with the following stages in common: a requirements stage, a specification phase, an implementation phase, a testing phase, and a maintenance phase. Although the model has been ridiculed and used as an example to avert, in that location is a general consensus that the tasks described in these phases remain of import in any software development effort. Most of the models nosotros have seen since, such as those in iterative, incremental, and spiral approaches, relate to the tasks identified in the original waterfall method.

One item trend that has been influential in recent years is the inclusion of insights on human being psychology and interaction patterns into the software development process. This has led to the development of so-called agile approaches [12], among which Scrum [25] is specially pop and successful.

Software Quality Models

The goal of any software evolution process is to promote the quality of the software under development. The notion of software quality is, however, highly nontrivial and it has therefore been field of study to significant enquiry and standardization efforts. These efforts take a common starting point in the efforts of McCall et al. [eight, 22] and Boehm et al. [6], where quality aspects are ordered into a bureaucracy. Figure 10.1 depicts the hierarchy as defined by Boehm.

Fig. 10.i

The hierarchy of software quality characteristics of Boehm et al. [6]. The edges in the hierarchy denote necessary atmospheric condition. For example, testability, understandability, and modifiability are necessary atmospheric condition for maintainability

Full size image

Later efforts refined and updated the underlying ideas of McCall and Boehm in several ways. In item, the quality attributes highlighted in these early on models have been argued to exist heavily based on the designer'south point of view. At that place are, still, other stakeholders, such as the user/customer and the visitor that develops the software every bit part of a business concern model. Depending on the function, the importance given each aspect may differ [4]. Security as a quality metric has also grown considerably in importance over the decades. It is thus included in all modern quality models and has been the subject of meaning academic scrutiny [xv].

Software Quality Management

Quality management has a long history in product and development. Nearly of the concepts developed accept been shown to be transferrable to software development, although with some adaptations. Alan Gillies [16] lists iv main aspects of quality management for software development:

Development procedures :

These include the tools used by the development team, in testing procedures, and in training staff.

Quality command :

This includes planning, progress meetings, documentation control, lawmaking walkthroughs, and so along.

Quality improvement :

Organized staff activity aiming at establishing a quality civilisation amongst the staff.

Quality assurance :

Monitoring activity to ensure that all aspects of the quality organisation are carried out correctly.

In that location exist many different manifestations of these aspects in organizations, standards, and the academic literature. What they all have in common is that they specify the quality of work processes. This will, of grade, have implications on the quality of the finished product, but hard requirements of the production itself are to be plant elsewhere. We therefore motion on to the metrics of software quality.

Software Quality Metrics

Whereas a quality model as described in Sect. 10.three gives us a structure for the quality concept, the model has to exist supported past metrics to be useful in a quality assurance setting. Unfortunately, there is a divide between the qualitative notions in a model and our ability to accurately capture these notions in numbers. It is a sad fact that the quality of software cannot be measured in terms of absolute and unambiguous scales [xvi].

Depending on the quality notions under scrutiny, the definitions and collection of metrics differ significantly in complication. Whereas portability is a quality characteristic that tin largely be captured by the frequency and number of platform-dependent constructs in the code, the notion of user friendliness is typically captured with metrics involving questionnaires and definiteness/robustness tin can be captured by logging the number of post-release bugs corrected.

The main interest of this book concerns the metrics that chronicle to security. Since security has been included as a dissever quality characteristic in near quality models, the world has witnessed many attempts to capture security in a quantitative way. Notwithstanding, every bit pointed out by Jansen [21], we take a long way to go and it is not likely that all aspects of the problem are resolvable. One reason for this is that our understanding of security does not degrade gracefully with the number of security flaws in the code. With one flaw, all security could be lost and the second flaw thus adds very fiddling to the situation. Although this is not true in all scenarios, it is clearly true in ours. If a organisation vendor has included a hidden backstairs in the lawmaking, the number of backdoors included in full is of picayune significance.

Standards

The notion of quality direction is intrinsically elusive, since it has different meanings for different people, in different roles, in different situations, and at different times. This issue presents problems in whatever manufacture that needs to be able to communicate with some degree of accuracy and efficiency how quality is ensured in an organization or for a product. A series of standards are being developed to address these problems and some of the ones used in the development of software are listed below [15].

ISO 9001 :

This is a general standard applicable to whatsoever system in any line of business. It defines a prepare of operation processes and proposes designing, documenting, implementing, monitoring, and continuously improving these operation processes [17].

ISO/IEC 9126 :

This standard contains a quality model for software and a set of metrics to support the model. It was showtime issued in 1991 [20].

ISO/IEC 25010 :

In 2011, this standard replaced ISO 9126. It contains a modernized ready of quality attributes and, in detail, attributes related to security were given more than attending [18].

ISO/IEC 15504 :

This standard, which is sometimes referred to as SPICE, covers a broad set up of processes involved in software acquisition, development, performance, supply, maintenance, and back up.

CMM/CMMI :

The term CMM stands for Capability Maturity Model and specifies software evolution processes. It is structured such that a development organization can be defined as belonging to 1 of a set of maturity levels, depending on the processes it has in place. The term CMMI stands for Adequacy Maturity Model Integration, which superseded CMM [10].

ISO 27001 :

This model was specifically designed for the certification of information security processes and is part of a series of standards related to information security [19]. Like ISO/IEC 25010, ISO/IEC 15504, and CMM/CMMI, ISO 27001 is procedure centric. This means that the models are based on the underlying supposition that if the procedure is correct, the outcome will be satisfactory [16].

The use of such standards and their certification arrangements is likely to have had positive effects on the quality of software. Information technology is also reasonable to assume that the standards that focus on security aspects, such as the ISO-27000 series and Common Criteria, have contributed to the development of secure software and to the prophylactic performance of information and communications applied science installations. However, it is important to note that the contribution of these standards lies in the structured use of methods that take been described and developed elsewhere. Therefore, if our problem – namely, verifying that an equipment vendor is indeed performing malicious acts against the states – cannot exist solved past existing technical approaches, and so structuring these approaches into a standardized framework is not likely to help. Neither is the certification of an organization or a piece of software co-ordinate to the same standards. In our quest for a solution, we must therefore await at the strengths of our technical methods. Only when they are strong enough can nosotros benefit from making them into a standard.

Common Criteria (ISO/IEC 15408)

The Common Criteria [ane] is the result of a joint effort by many nations. They comprise a standard that requires our item attention for two reasons. Kickoff, like ISO 27001, it is aimed specifically at security. Second, unlike ISO 27001, information technology contains requirements for products, non only processes.

The cardinal concepts in this standard are the protection profiles by which a user specifies security requirements. A vendor claiming to accept a production meeting these requirements tin accept the production tested for compliance in contained accredited testing laboratories. The rigour of the testing tin can be adapted to the criticality of the component in question and, for this purpose, vii Evaluation Assurance Levels (EALs) have been divers. The everyman of these, EAL1, requires just functional testing for correct operation, whereas EAL7 mandates full formal verification of the source lawmaking:

• EAL1: Functionally tested.

• EAL2: Structurally tested.

• EAL3: Methodically tested and checked.

• EAL4: Methodically designed, tested, and reviewed.

• EAL5: Semiformally designed and tested.

• EAL6: Design semiformally verified and tested.

• EAL7: Pattern formally verified and tested.

The wide adoption of Mutual Criteria has been a major stride towards more secure it systems. Still, nix in this effort addresses cases in which the company developing the organization injects unwanted functionality. The highest assurance level, EAL7, is, at the time of writing, considered prohibitively costly for virtually evolution projects for reasons discussed in Chap. 9. Furthermore, evaluation focuses primarily on assessing documentation, which ultimately has to exist provided by the visitor that is not to be trusted, and secondarily on functional testing, which we concluded in a higher place can be hands fooled. Then, at the cease of the solar day, nosotros have to describe the same conclusion for Common Criteria as we did for the other standards. Structuring existing methods into a framework will not solve our problem unless there is a solution in the methods themselves. In the following sections, we cover the basic methods that make upwardly the assurance levels of Common Criteria.

Software Testing

Testing remains the virtually oftentimes used technique for the validation of the functional quality of a slice of software. It has been claimed that, in a typical evolution projection, more than 50% of the total price is expended in testing and this figure has remained more or less stable since the early days of programming [24]. The importance of testing in the evolution of stable and trustworthy software has led to a large body of research in the field. Still, we have non been able to turn testing into an exact science in the sense that there exist exam methodologies with the guaranteed ability to find all faults. In that location are several reasons for this.

First, for most software of any complexity, the combination of stimuli that the software should be able to handle is subject area to a combinatorial explosion that prevents the tester from exhausting all possible examination scenarios. This means that an important part of the testing effort is to choose the set of tests the software should undergo [ii]. The goal of this test set could exist to cover all branches of the code or – for some definition of input equivalence – all the equivalence classes of input parameters.

Nearly serious testing situations involve such a large set of test cases that they volition have to be carried out automatically. This brings usa to the 2nd problem of testing, namely, that of automatically recognizing incorrect behaviour. In the field of software testing, this is oftentimes referred to every bit the test oracle trouble [3]. In circuitous applications, this trouble is frequently a major obstacle in the testing process.

Some software systems are really accounted untestable. These are systems where verbal test oracles cannot be defined or where the organisation interacts with the physical surround in such complex ways that information technology is incommunicable to fairly cover all the test cases. For these cases, edifice a model of the system and then performing tests on the model instead of the software itself has been proposed [seven]. This concept is quite similar to model checking, which is discussed in Sect. 10.nine.

Although software testing is and remains the main vehicle for the validation of software functionality, the state of the art falls brusque of helping us when facing a potentially quack equipment vendor. Triggering unwanted software tin be accomplished past arbitrarily complex sets and sequences of stimuli, meaning that the likelihood of being detected in a test case can be fabricated arbitrarily small. Furthermore, model testing is non likely to help u.s., since the malicious functionality volition clearly not be communicated in a way that will let it become office of the model. For testing to exist of help to u.s.a., we will need a scientific quantum that is not on the horizon at the time of this writing.

Verification Through Formal Methods

One of the methods referred to in the Mutual Criteria is code verification. This consists of writing an authentic specification of how the software should conduct and performing a rigorous mathematical proof that the piece of software actually conforms to the specification. The correctness of this proof can in plow exist checked automatically by a relatively simple plan. In Chap. 9, we discussed formal methods in some detail and concluded that both promise and despair are associated with this approach.

The hope is related to the fact that the arroyo itself carries the promise of being watertight. If at that place is a formal and verifiable proof that in that location are no malicious actions in the software code, then a mathematical certainty has been established. The despair is due to ii facts. Showtime, it tin be argued that formal methods simply move the complexity from understanding the software to agreement and proofreading the specifications. The second – and this is the more serious one – full formal specification and verification are considered prohibitively costly for larger systems [11]. This is farther supported by the fact that very few pieces of software have been reported to accept undergone full formal verification equally specified in the highest balls level, EAL7, of Mutual Criteria.

One particular evolution that seems to address both the complexity of specification and the cost of formal verification is model checking [9]. This consists of building an abstract model of the software that is sufficiently small then that its state infinite can be made the subject field of an exhaustive search. Although this is a promising arroyo, our problem pops upwards again in the question of how to build the model of the software. This task is intrinsically difficult and has to be carried out by human experts [5]. The likelihood of an intentional backdoor in the code showing upward in the model is therefore quite similar to the likelihood of information technology being detected by reviewing the code itself. Model checking is therefore non a silvery bullet to our problem, and so we move on to seeing if code review can help.

Code Review

Lawmaking reviews have been performed in security engineering for a long time. The goal of code reviews in security is to await for known classes of vulnerabilities that typically issue from programming mistakes. These reviews can be performed as function of the development process or take place entirely subsequently the system has been developed. In that location are several types of reviews, depending on which code is available. Downwards et al. [11] mention the following alternatives, varying in the information bachelor to the tester:

• Source only.

• Binary only.

• Both source and binary.

• Binary with debugging information.

• Black box, where only the running application is available.

Every bit argued in Chap. iv, a review of the source code lone gives a dishonest equipment vendor full opportunity to include malicious code through the compiler or other development tools. In the example of the black box, we are, in principle, left with testing, a state of affairs we discussed in Sect. 10.8. This leaves us with the testing situations in which the binaries are available.

Lawmaking reviews of binaries have many the aforementioned strengths and weaknesses equally formal methods. On one paw, they hold the promise that, when carried out flawlessly and on a complete binary representation of a piece of software, all hidden malicious code can exist constitute. Unfortunately, at that place are 2 major shortcomings: ane is that finding all hidden malicious code is by and large impossible, every bit shown in Chap. 5. Even finding some instances volition be extremely challenging if the malicious code has been obfuscated, as described in Chap. 7. The second shortcoming is ane of scale. Down et al. [11] judge that an experienced code reviewer tin analyse between 100 and one,000 lines of code per twenty-four hours. Morrison et al. [23] approximate that a full analysis of the Windows code base should take betwixt 35 and 350 person–years. To make things worse, these figures are based on the assumption that information technology is the source code that is to be analysed, not the binaries; the real costs are therefore likely to exist college.

One way to handle this problem is to perform a code review on only parts of the software product. Which parts are to exist selected for scrutiny could be called by experts or by a model that helps cull the parts that are more probable to contain malicious lawmaking. Such defect prediction models have been used for some time in software engineering to look for unintended faults and a range of similar models take been proposed for vulnerability prediction. The value of the vulnerability prediction models is currently under debate [23]. Nosotros do not need to take a stand in that give-and-take. Rather, we find that the models effort to identify modules that are more than probable to contain unintended vulnerabilities. If the vulnerabilities are intended, however, and we tin can presume that they are hidden with some intelligence, then a model is not likely to be able to place the hiding places. Even if such a model existed, it would have to be kept secret to remain constructive.

It is thus a deplorable fact that rigorous lawmaking reviews and vulnerability detection models likewise leave ample opportunity for a dishonest equipment producer to include malicious code in its products.

Discussion

Fuggetta and Di Nitto [13] took a historical view of what was perceived every bit the most important trends in software procedure enquiry in 2000 and compared this to what was arguably most important in 2014. One of the fundamental observations is that security has gained significantly in importance and, consequently, the community should 'place and assess the major issues, suggest means to monitor and manage threats, and assess the impact that these problems tin can take on software development activities.' Although the statement was intended for scenarios with unintended security holes and where the perpetrator is a tertiary party, it does shed light on the extent to which software development processes and quality assurance are promising places to look for solutions to the problem of untrusted equipment vendors and system integrators. If these tools and methods are inadequate to help the developers themselves notice unintended security holes, they are probably be a far weep from helping people exterior of the development squad to find intentional and obfuscated security holes left there by a dishonest production developer.

The findings in this chapter ostend the above observation. We have covered the concepts of evolution processes, quality models, and quality management and have found that they relate too indirectly to the lawmaking that runs on the machine to qualify every bit adequate entry points in the verification of absenteeism from deliberately inserted malware. The field of software quality metrics in some aspects direct relates to the source code only, unfortunately, it is doubtful that all aspects of security are measurable. In addition, counting the number of bugs found has still not given us bug-free software; thus, it is not likely that counting security flaws will provide whatsoever guidance in finding deliberately inserted malware.

Methods that go deep into the code running on the car include testing, lawmaking reviews, and formal methods. In the present state of the art, formal methods do not scale to the programme sizes we crave, code review is fault prone and too fourth dimension-consuming to be watertight, and testing has limited value because of the problem of hit the right test vector that triggers the malicious behaviour. Even when triggered, recognizing the malicious behaviour may be highly nontrivial. It is clear that these methods still leave ample space for dishonest equipment providers to insert unwanted functionality without whatsoever real danger of beingness exposed.

Although these approaches still autumn far short of solving our problem, they remain a viable starting point for research on the topic. In particular, we hope that, at some time in the time to come, the combination of formal methods, careful binary code review, and testing volition increase to a significant level the risk of being exposed if y'all include malware in your products. However, we have a long way to become and a great deal of research must exist done before that point is reached.

References

1. http://www.commoncriteriaportal.org

2. Ali, Southward., Briand, L.C., Hemmati, H., Panesar-Walawege, R.K.: A systematic review of the application and empirical investigation of search-based test example generation. IEEE Trans. Softw. Eng. 36(6), 742–762 (2010)

   CrossRef  Google Scholar

3. Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The oracle problem in software testing: a survey. IEEE Trans. Softw. Eng. 41(5), 507–525 (2015)

   CrossRef  Google Scholar

4. Berander, P., Damm, L.O., Eriksson, J., Gorschek, T., Henningsson, 1000., Jönsson, P., Kågström, S., Milicic, D., Mårtensson, F., Rönkkö, Grand., et al.: Software quality attributes and trade-offs. Blekinge Plant of Applied science (2005)

   Google Scholar

5. Bérard, B., Bidoit, 1000., Finkel, A., Laroussinie, F., Petit, A., Petrucci, L., Schnoebelen, P.: Systems and Software Verification: Model-Checking Techniques and Tools. Springer Science and Business organization Media, Berlin (2013)

   MATH  Google Scholar

6. Boehm, B.W., Brown, J.R., Kaspar, H.: Characteristics of Software Quality. North-Holland, Amsterdam (1978)

   MATH  Google Scholar

7. Briand, L., Nejati, Due south., Sabetzadeh, M., Bianculli, D.: Testing the untestable: model testing of complex software-intensive systems. In: Proceedings of the 38th International Conference on Software Technology (ICSE 2016), ACM (2016)

   Google Scholar

8. Cavano, J.P., McCall, J.A.: A framework for the measurement of software quality. In: ACM SIGMETRICS Functioning Evaluation Review, vol. 7, pp. 133–139. ACM (1978)

   Google Scholar

9. Clarke, E.M., Grumberg, O., Peled, D.: Model Checking. MIT Press, Cambridge (1999)

   Google Scholar

10. CMMI Product Team: CMMI for development, version ane.2 (2006)

    Google Scholar

11. Down, Thou., McDonald, J., Schuh, J.: The Fine art of Software Security Assessment: Identifying and Preventing Software Vulnerabilities. Pearson Education, Boston (2006)

    Google Scholar

12. Fowler, M., Highsmith, J.: The agile manifesto, Softw. Dev. 9(8), 28–35 (2001)

    Google Scholar

13. Fuggetta, A., Di Nitto, Eastward.: Software process. In: Proceedings of the on Future of Software Engineering, pp. i–12. ACM (2014)

    Google Scholar

14. Galin, D.: Software Quality Assurance: From Theory to Implementation. Pearson education, Boston (2004)

    Google Scholar

15. García-Mireles, G.A., Moraga, K.Á., García, F., Piattini, Thou.: Approaches to promote product quality inside software process improvement initiatives: a mapping study. J. Syst. Softw. 103, 150–166 (2015)

    CrossRef  Google Scholar

16. Gillies, A.: Software Quality: Theory and Management (2011). https://www.lulu.com

17. ISO 9001:2015 quality direction systems - requirements

    Google Scholar

18. ISO/IEC 25010:2011 systems and software engineering – systems and software quality requirements and evaluation (square) – system and software quality models

    Google Scholar

19. ISO/IEC 27001:2013 data engineering science – security techniques – information security management systems – requirements

    Google Scholar

20. ISO/IEC 9126-1:2001 software engineering science–product quality–part 1: quality model

    Google Scholar

21. Jansen, W.: Directions in Security Metrics Inquiry (2009)

    Google Scholar

22. McCall, J.A., Richards, P.Yard., Walters, 1000.F.: Factors in software quality. volume I. Concepts and definitions of software quality. Technical report, DTIC Document (1977)

    Google Scholar

23. Morrison, P., Herzig, M., White potato, B., Williams, L.: Challenges with applying vulnerability prediction models. In: Proceedings of the 2015 Symposium and Bootcamp on the Science of Security. ACM–Association for Computing Machinery (2015)

    Google Scholar

24. Myers, Grand.J., Sandler, C., Badgett, T.: The Art of Software Testing. Wiley, New York (2011)

    Google Scholar

25. Schwaber, K., Beedle, M.: Agile Software Evolution with Scrum. Prentice-Hall, Englewood Cliffs (2001)

    Google Scholar

Download references

Author data

Affiliations

1. Simula Research Laboratory, Fornebu, Norway

   Olav Lysne

Corresponding author

Correspondence to Olav Lysne .

Rights and permissions

This chapter is published under an open access license. Please cheque the 'Copyright Information' section either on this page or in the PDF for details of this license and what re-employ is permitted. If your intended utilise exceeds what is permitted by the license or if you are unable to locate the licence and re-utilise information, please contact the Rights and Permissions team.

Copyright information

© 2018 The Author(due south)

Most this affiliate

Cite this chapter

Lysne, O. (2018). Software Quality and Quality Management. In: The Huawei and Snowden Questions. Simula SpringerBriefs on Computing, vol four. Springer, Cham. https://doi.org/ten.1007/978-3-319-74950-1_10

Download citation

• .RIS
• .ENW
• .BIB

• DOI : https://doi.org/10.1007/978-3-319-74950-1_10

• Published: 20 Feb 2018

• Publisher Name: Springer, Cham

• Print ISBN: 978-three-319-74949-5

• Online ISBN: 978-3-319-74950-1

• eBook Packages: Figurer Scientific discipline Informatics (R0)

mcbrydejoher1959.blogspot.com

Source: https://link.springer.com/chapter/10.1007/978-3-319-74950-1_10

Share :