The Nature of Software Engineering

Questions naturally arise about what software engineering actually entails. How do we qualify and evaluate software engineers? How do we validate their experiences? A first reaction may be to approach these tasks in the same way that we have done for all other engineering disciplines. However, software engineering differs from structural, mechanical, and electrical engineering in subtle ways. The differences are linked to the soft, but rather unkind, nature of software. In this article, four key differentiating characteristics are explored:

• Absence of a fundamental theory


• Ease of change


• Rapid evolution of technologies


• Very low manufacturing costs

Absence of a Fundamental Software Theory

Despite all the research done by computer scientists, there is no equivalent in software for the fundamental laws of physics. This lack of theory, or at least the lack of practically applicable theories, makes it difficult to do any reasoning about software without actually building it. During design, software can be structured and partitioned into chunks, but the real thing (once it crawls inside a computer) is actually totally unstructured, so that anything that goes wrong somewhere can corrupt something somewhere else. The absence of solid and widely applicable theory also means that the few software engineering standards we do have rely on good practice alone, whereas building codes in other disciplines can trace their rules to sound physical principles.


Ease of Change

Software is, almost by definition, easy to change, so naturally, organizations want to take advantage of this characteristic. There is pressure to change software throughout its entire development and even after it's delivered. If you're building a bridge, you don't have this kind of flexibility. You cannot say, "Hmm, now that I see the pilings, I would like this bridge to be two miles upstream." But it is very, very difficult to change software in a rigorous fashion, with all ramifications of all changes fully understood and completely coordinated. Again, because of the absence of solid theory, it's hard to validate a change set and its impact without actually doing all the changes. Most of the damage that is done to software is done through changes.


Rapid Evolution of Technologies

Software development techniques, and the environment of software itself, are changing at an extremely rapid pace that does not allow for progressively consolidating a body of knowledge. This puts a lot of pressure on companies to train and re-train their software engineers, and some do not really understand why they have to spend four times more per capita on training than do people from other disciplines. This also makes the initial training software engineers have received less important. This rapid evolution also means that it is more and more difficult to maintain and evolve "legacy" systems -- because the technologies used some ten years ago are not in use any more, and people who still have mastery over them are rather rare. The norms and standards also have to evolve rapidly to catch up with technology evolution. Finally, software engineering, unlike other disciplines, has not had the benefit of hundreds or thousands of years of experience.


Very Low Manufacturing Cost

First, note a slight shift in paradigm. When software engineers speak about design, by this they mean only a high-level description of their intent, and then they think of program construction as akin to manufacturing. Actually it is not -- program implementation is more like preparing a cast in mechanical engineering. Also, for a software engineer, a prototype is roughly equivalent to a scale model; it's pretty incomplete. The real "manufacturing" of software entails almost no cost; a CD-ROM, for example, costs less than a dollar, and delivery over the Internet only a few cents. Often it doesn't matter if the design -- that is, the initial program -- is a bit wrong; we can just fix it and manufacture it again, as we noted above in the discussion on ease of change. Clearly, this combination -- ease of change and low manufacturing cost -- has led the software industry into a pretty big mess. And these practices are supported by outrageous licensing policies that allow the designer and manufacturer to assume no responsibility other than, in good cases, a promise that they will re-manufacture the product in a few days or months. You can't do that with a bridge or a car engine because the cost would be huge, and that forces engineers involved in building these things to get them right the first time.


Engineering All the Same?

For twenty years, refusing to acknowledge the four factors I described, the software industry has tried hard to pretend that software development could follow the same path as other engineering disciplines. We have failed. We also hoped that science would bring us solutions, but they have not been forthcoming. Why? To answer that question, let's look at two important ways that software engineering departs from other disciplines.

Iterative Development

The very rational and straightforward "waterfall" development lifecycle -- define and freeze requirements, create and validate the design, implement and test, then deliver -- works very well in many disciplines but has failed many times in software engineering. This project lifecycle does not accommodate changes: It does not allow you to really validate much, so you have to rely on your own warm, fuzzy feeling that "the design is OK." Nor does it allow for tactical changes in technology, or take advantage of the low manufacturing cost -- except for pushing undue costs on to consumers.

Today, software engineers take a more iterative approach to software development, which allows them to integrate changes, to refine and validate the design based on execution and not just examination, and to accommodate evolution in technology. An iterative approach would be impossible in other disciplines; you cannot build a bridge iteratively, for example.

Iterative development allows you to continuously verify the quality of a constructed prototype as opposed to demonstrating correctness a priori, based on fixed laws. Software has no laws that can ensure the ultimate product will perform as expected, but iterative development allows you to confront technical risks earlier in the development cycle.

Also, software engineering puts more emphasis on some techniques that have less importance in other disciplines, such as requirements management and change management, because requirements and other software artifacts may change throughout the development lifecycle and even after that.

Component-Based Development

Another dream of software engineers is to mimic with software what has happened in electronics or construction -- to develop families of standardized parts out of which you can build larger and larger sub-assemblies and ultimately complete systems. This sounds straightforward, but actually very few can do it. Again, this is due to the lack of a strong underlying theory to rigorously define the components and their interface. The situation has not been helped by the rapid changes in technologies: No component family has had time to settle and develop as a self-sustaining industry. Only recently has a new sub-discipline emerged -- software architecture -- which tries to address, despite the lack of fundamental theory, some of the structural aspects involved in constructing large software programs and reusing software assets across product lines or product families.

So, if software engineering should be treated on an equal footing with other engineering disciplines and not solely as computer science or some kind of enlightened craftsmanship, then we also have to acknowledge fundamental differences that make some of the more traditional approaches to engineering and engineering management inapplicable to software. Software engineering has developed its own approaches to manage its specificities over the last few years.

I believe that the registration process for professional software engineers -- as well as the construction of accredited software engineering programs -- must understand, acknowledge, and address these specificities.


About Author:
- Philippe Kruchten, former director and general manager of the IBM Rational Software Process (RUP) Business Unit, http://www.ibm.com/developerworks/rational/library/4700.html.