Making systems

Table of contents

• Part V: Development methodology and life cycles
• Chapter 21: Introduction to life cycle patterns
• 21.1 Introduction
• 21.2 Life cycle and development methodology
• 21.3 Key ideas
• 21.4 Purpose of life cycle patterns
• 21.5 A model for patterns
• 21.6 Documenting life cycle patterns
• 21.7 Work steps and artifacts
• 21.8 Life cycle and teams
• 21.9 Life cycle and planning
• 21.10 Principles for a life cycle pattern
• Chapter 22: Development methodologies
• 22.1 Purpose
• 22.2 Methodologies
• 22.3 Practical considerations
• Chapter 23: Example life cycle patterns
• 23.1 Introduction
• 23.2 Whole project life cycle
• 23.3 System development patterns
• 23.4 Post-development patterns
• 23.5 Detail patterns
• 23.6 Comparisons and lessons learned
• Part VI: Reference life cycle
• Chapter 24: Introduction
• 24.1 Projects with proposals
• 24.2 Project-wide decisions and reviews
• Chapter 25: Project preparation
• Chapter 26: Project support
• Chapter 27: Development
• 27.1 What is development?
• 27.2 The development phase
• 27.3 Purpose development
• 27.4 Concept development
• 27.5 Development methodology
• 27.6 System feature development
• 27.7 Recursion to component development
• 27.8 Feature development variations
• 27.9 Acceptance
• Chapter 28: Operation
• 28.1 System production
• 28.2 System production examples
• 28.3 System deployment
• 28.4 System deployment examples
• 28.5 System operation
• 28.6 System operation examples
• 28.7 System evolution
• 28.8 System evolution examples
• 28.9 System retirement
• 28.10 System retirement examples
• Chapter 29: Project ending
• 29.1 Project cancellation
• Chapter 30: Using the reference life cycle
• 30.1 Meeting life cycle principles
• 30.2 Relation to NASA life cycle
• Part VII: Specifications
• Chapter 31: System purpose
• 31.1 Introduction
• 31.2 Investigating purpose
• 31.3 Recording purpose
• 31.4 Changes of purpose
• Chapter 32: System concept
• 32.1 Copied in
• 32.2 Purpose of concept development
• 32.3 Concept development work
• 32.4 When the concept is complete
• 32.5 Evolving the concept
• 32.6 Concept development for different kinds of projects
• 32.7 Artifacts
• Chapter 33: Specifications
• 33.1 Purpose
• 33.2 Specifications and systems
• 33.3 Example
• 33.4 Kinds of specifications
• 33.5 Using specifications in a system
• 33.6 Using specifications for a component
• 33.7 Specification artifacts
• Chapter 34: Requirements
• 34.1 What are requirements?
• 34.2 A single requirement
• 34.3 Multiple requirements
• 34.4 Organizing requirements
• 34.5 Writing good requirements
• 34.6 Requirement derivation
• 34.7 Advanced requirements
• 34.8 Analyzing requirements
• 34.9 Validating requirements
• 34.10 Verification
• 34.11 Limitations of requirements
• 34.12 Working with requirements
• 34.13 Tools
• Chapter 35: Models
• Chapter 36: Interface definitions
• Part VIII: Design
• Chapter 37: Design introduction
• 37.1 Purpose
• 37.2 How designs are used
• 37.3 Design artifacts
• 37.4 Developing designs
• Chapter 38: Breakdown structure
• 38.1 What is the component breakdown for?
• 38.2 Basic concepts
• 38.3 Component identifiers
• 38.4 Viewing the breakdown structure
• 38.5 Context and relationships
• 38.6 Advice
• 38.7 Examples
• Chapter 39: Component design
• 39.1 Form
• 39.2 State
• 39.3 Actions or behaviors
• 39.4 Interfaces
• 39.5 Non-functional properties
• 39.6 Environment
• Chapter 40: Good design practices
• Chapter 41: Prototyping
• Chapter 42: Budgets
• Chapter 43: Safety and security design
• Part IX: Implementation
• Chapter 44: Introduction
• Chapter 45: Good implementation practices
• Chapter 46: Integration-first implementation
• Chapter 47: Skeletons and mockups
• Chapter 48: Implementation using testing
• Part X: Verification
• Chapter 49: Introduction
• Chapter 50: Recording evidence
• Chapter 51: Good verification practices
• Bibliography

Part V: Development methodology and life cycles

What life cycles and development methodologies are. It covers what goes into a life cycle pattern and how the patterns relate to other parts of how a project operates. It defines development methodologies and how these relate to life cycles. The last chapter lists several example life cycle patterns, which lead into a comprehensive reference life cycle in Part VI.

Chapter 21: Introduction to life cycle patterns

21.1 Introduction

System building in general follows a common story.

A project to develop a new system begins when someone has an idea that people should make the system. At this initial moment, the system is largely undefined. There is a vague concept in a few minds, but all the details are uncertain.

The project then moves the system from this initial concept through to an operational system, and through the system’s operational life and eventual retirement. During development, the team will need to ensure steps are taken in order to produce a correct, safe system. Designs will be checked. Implementations will be tested. The system as a whole will be verified before being deployed into service. At the same time, the resources spent on building the system must be used efficiently, doing the work that needs to be done and avoiding the work that doesn’t need to be done.

Many projects continue system development beyond the first operational version, with ongoing development or problem fixes. Some projects include the steps to shut down and dispose of the system once it has completed its functions.

The life cycle is how a project organizes the way the team moves through this story. It is a pattern that defines the phases and steps in the work: what will come first, what will done before something else, when checks will happen. It provides checklists to know when some step is ready to be done, and when it should wait for prerequisites. It provides checkpoints and milestones for reviewing the work, so that problems are found and dealt with in a timely way. It provides an overall checklist to ensure that all the work that needs to be done is in fact done.

Section 20.3 introduced the basic ideas for life cycle patterns. These include:

• The life cycle usually includes multiple patterns for different parts of the work.
• Work is organized as a set of phases or steps, with milestones or checkpoints included in some steps. (I use the terms phase and step interchangeably; I generally think of a “phase” being longer than a “step”.)
• Phases or steps can be dependent on each other, with A depending on B meaning that work done in A will build on work done in B.
• A phase can have conditions that should be met before it starts, or that should be true when it is complete.
• Many phases and steps can be worked on in parallel.
• A path from one step to another following dependencies does not necessarily mean that the steps must be performed serially.

Each project will use its own life cycle patterns. The patterns may incorporate a framework that is standard for the industry or the parent organization. Selecting and documenting the patterns is an essential part of starting up a project, and people in the project should review how well the patterns are working for them from time to time and may want to improve the patterns.

In this part, I discuss life cycles in general. In Part VI, I present a reference life cycle pattern.

21.2 Life cycle and development methodology

Life cycle patterns are related to, but separate from, the development methodology that a team chooses to use, such as waterfall, spiral, or agile methodologies. I address these methodologies in the next chapter (Chapter 22).

Speaking broadly, the development methodology determines how the work is organized in time: in a single sequence or iteratively, synchronized tasks or separate tasks, how far ahead to plan. The life cycle patterns reflect some of those methodology decisions and encode how to do different tasks.

Put another way, the life cycle patterns help organize what work the project has to do, and what dependencies there are among different steps in the work. The development methodology organizes how that work is planned and scheduled. As a result, the two go hand-in-hand but are distinct from each other.

21.3 Key ideas

Almost all project life cycle patterns, for both whole systems and for components, follow a similar overall flow. Abstracting from the story in the introduction, there are phases:

1. Working out how the project will operate
2. Identifying purpose
3. Developing a concept
4. Refining concept into specification and design
5. Implementation
6. Verifying the result
7. Operating the system or component
8. Evolving it over time
9. Retiring the system or component at end of life
10. Shutting down the project

For a whole system, this looks like:

Note that this flow starts with the system or component’s purpose. Good engineering always begins with having a clear understanding of what a thing is for. I have watched many engineers rush into designing and building a component without putting time into understanding what the component is going to be used for. By random chance their design has occasionally worked out to match what the component actually needed to do, but only rarely.

Understanding a system’s purpose or a component’s purpose also provides a way to bound the work. If one doesn’t know what a component is for, it is easy to keep working on a design without stopping because there isn’t a clear way to know when the design is good enough to be called done.

There are many points in this flow where one might add checks. At these times one can check on the correctness of the work. These checks improve system quality by building in the opportunity to discover and correct flaws before other work builds on the flawed work. Finding minor problems quickly usually means the cost of correction remains low.

There are also points where a project might have project-wide decisions--go/no go decisions or key decision points. These provide opportunities to check the entire project progress, sometimes occurring in the middle of other work, or at times when irrevocable actions are to be taken, such as funding, launch, or public announcements.

This general pattern applies recursively. One can start by creating a specification and design for the system. The system design will decompose the system into high-level components (Section 6.4). The act of defining a set of components implies identifying a purpose for each one, then specifying and designing each high-level component. The design of a high-level component might in turn decompose into a set of lower-level components, which in turn need a purpose, then specification and design.

The overall flow shows a move from high uncertainty at the beginning to lower uncertainty as the work proceeds. I will address managing using uncertainty in ! Unknown link ref.

Finally, a project’s life cycle patterns will reflect the development methodology that the team has selected. Waterfall, spiral, and agile development all affect the contents of the patterns. I discuss this more in Chapter 22.

The life cycle is provides a general set of patterns for how work should proceed, but it should not define exactly how each work step should be done. That is left to procedures (Section 20.4), which should provide step-by-step instructions for how to do key parts of the life cycle. For example, if a life cycle phase indicates that a design review and approval should occur before the end of a design phase, then there should be a corresponding procedures for design reviews. That procedure should indicate who should be involved in a review, what they should look for, how those people will communicate about the results, who is responsible for approving the design, and how they indicate approval.

The life cycle patterns are the basis for the project’s plan (Section 20.5). The patterns are a set of building blocks that people in the project can use to develop the plan. The plan, in turn, guides tasking: the selection of which tasks (as defined in the plan) people should be working on next.

21.4 Purpose of life cycle patterns

Life cycle patterns address problems that projects have. They can help the team have a predictable and reproducible flow to how work should be done, so that everyone shares the same understanding of how the team works.

There are six ways that life cycle patterns help a project.

1. Quality of work. The team must build a system that addresses the customer’s purpose, and in doing so must meet quality, safety, security, and reliability objectives.
2. Efficiency. The project will be expected to deliver the final system as quickly as possible, at the lowest reasonable cost, while meeting the quality objectives. This means that the team needs to be kept busy doing useful work.
3. Team effectiveness. People on the team need to know how to work together. Building trust depends, in part, on having shared expectations of how each person will do their work.
4. Management support. Project management will need to plan and track the work in order to ensure the team meets deadlines and that they have sufficient resources to do the work.
5. Customer and regulatory support. The customer may have specific milestones they expect the project to meet in support of the customer’s acquisition processes. Regulators often have similar expectations if a system must be certified or licensed for operation.
6. Auditing support. The project’s work may be audited to check that the processes followed meet regulatory requirements, certification requirements, or as part of a legal review.

Gaining these benefits is not a result of using life cycle patterns per se; rather, it comes from using patterns that are designed to provide the benefits. For example, if the customer has an acquisition process that specifies certain milestones, then the top-level life cycle pattern for the project should incorporate those milestones. If the project is likely to have auditing requirements, then the patterns should include tasks to generate and maintain auditing records.

21.5 A model for patterns

Each project will have several life cycle patterns, each covering a different part of the work.

Each pattern is defined by its purpose, the circumstances in which it applies, the phases or steps involved, and the dependencies among the steps. It should also include rationale that explains why the pattern is structured the way it is. In a previous chapter I used the example of a simple pattern for building one component:

This pattern applies to building one low-level component where the purpose of the component is already known, and the component is straightforward to design and build in house. Similar but slightly different patterns might apply when the component has to be prototyped before deciding on a design, or when the component is being acquired from a supplier outside the project. This pattern would be used as one part of a larger pattern for building a higher-level component that includes this one.

Each phase of a pattern defines a way to move part of the work forward. It should have a defined purpose that defines what work should be achieved in that phase.

The details of the phase are defined by:

• Preconditions: what conditions should be true when a phase is ready to start. A list of conditions for starting the phase, beyond the artifacts that should be ready.
• Input artifacts that should be available at the beginning of the phase, and their maturity or completeness level.
• Actions that are to be taken to begin the phase.
• What work should be done in the phase, and the artifacts developed in the phase.
• Milestones to be met during the phase.
• Actions taken to end the phase.
• Output artifacts that should be available at the end of the phase, and how complete each one should be.
• Postconditions that define what should be true when the phase has completed.

Each action should also indicate who is responsible for performing that work. The responsibility will usually be defined as a role, not a specific individual. For example, a component design phase might involve three actions: design the component, review the design, and approve the design. The design action would be the responsibility of the component developer; the review action would be the responsibility of the developers responsible for components that interact with the one being designed, and the approval would be the responsibility of a systems engineer overseeing some higher-level component of which this one is part.

The rationale for this example design phase might say:

• The purpose is to work out a design for the component that meets its specifications, including its relationships with other components in the system structure and its safety, security, reliability, and performance objectives.
• A separate design step between specification and implementation gives time to think specifically about this component, and to document its design for future developers who may need to revise the design.
• The review actions provide an independent check that should improve the likelihood of this component working correctly as part of the system by looking at the design from a point of view that has not been intimately involved in working out the design.

The actions defined for the phase should reference the procedures for doing those actions, when those procedures are defined. For the example design review action, the procedure might be:

• The component developer notifies the review group that their review is needed;
• Each reviewer in the review group acknowledges that they will provide the review;
• Each reviewer checks the design against the relationship that component has with their own components, and against the specification for the component being reviewed;
• Reviewers give feedback to the developer, indicating whether they are satisfied or not;
• When all reviewers are satisfied, they inform the developer.
• If one of the reviewers detects a more serious problem with the design that cannot be resolved by feedback to the developer, the reviewer should use another procedure to raise the issue up to a higher level.

The procedure might also name the tools to be used (an artifact repository for the design, a review workflow tool for the reviews).

21.6 Documenting life cycle patterns

A team needs clear documentation of the phases if they are to execute them properly. A team can’t be expected to guess at what they need to be doing, or how their work will be reviewed; it needs to be spelled out.

This documentation is assembled during the project preparation phase. The details are usually not completely worked out before any other work is begun; rather, “project preparation” more often proceeds in small increments, working out the rules shortly before the associated work begins.

Each life cycle pattern should have a purpose, and the steps or phases in the pattern should be checked that they can achieve that purpose (and that there is no extraneous work in the pattern).

A pattern should also have an explanation of when it applies and when it does not. For example, there may be multiple patterns for designing a component: one for a simple component that is built in house; one for a component that is outsourced to a supplier; one for a high-level component that is made up of several lower-level components; one for a component that requires investigation or prototyping before deciding on a conceptual approach to its design. All these patterns likely have a lot in common, but procuring an outsourced component will have contracting steps that an in-house component will not.

Someone using the documentation should be able to tell accurately whether they are using the correct version of the patterns. The life cycle patterns will be revised from time to time—as the team grows and as people find ways to improve how they work together. This means that the material that a user sees should indicate not just a revision number but have a clear indication of whether the version they are looking at is not longer current.

The form of the documentation is not as important as the content. It can be a written document. It can be made available electronically, with structured access and search capabilities (such as in a Wiki). Some companies offer tools that help define and document development processes or life cycle patterns, including definitions of phases. What matters is that each person who needs to use the documentation can do so conveniently and accurately.

21.7 Work steps and artifacts

Each phase or step has a number of artifacts that the team must develop. At the end of a phase, some of those artifacts need to be complete (allowing for future evolution), and others need to have reached some defined level of maturity. The work in a phase consists of the tasks that develop those artifacts.

I discussed artifacts in Chapter 17. The artifacts are the products of building the system, including the system being delivered as well as documentation of its design and rationale, records of actions taken during development, and information about how the project operates.

These artifacts are the inputs and outputs of the work specified in life cycle patterns (and the associated procedures). Using the component design step example, the work uses:

• The purpose and rough concept developed for the component;
• The specifications developed for the component;
• Documentation of the relations between this component and others, both functional and non-functional; and
• The purpose of the system and of higher-level components of which this one is part.

The design step produces:

• A design document for the component;
• Analyses of the design showing whether it will meet its specifications or not;
• Records of the rationale for why the design was chosen; and
• Records of review and approval of the design.

In general, every artifact involved in building the system should be a product of some work phase or step, and every input or output of work steps should be included in the set of artifacts the team will develop. Ideally, the life cycle patterns will be checked for consistency with the list of artifacts the project uses.

Artifacts are developed at different times during the course of a project. A few artifacts should be worked out as the project is started—especially those recording the initial understanding of the system’s purpose and initial documentation of how the project will operate. These will be refined over time. Other artifacts are developed during the course of development, and the life cycle patterns indicate which ones are to be worked out before others. The artifacts will be in flux during development: the team learns about the system as it designs and develops it; the customer or mission needs often change over time; flaws get discovered in designs or implementations.

Many of the project’s artifacts support how people work together, and the life cycle patterns should reflect these communication needs. For example, one person may work out the protocol that two components need to use to communicate with each other. Two other people may design and implement the two components. The interface specification that the first person develops serves to communicate the details of the interface among all three people. The patterns should record that the design and implementation work steps depend on the work to develop the interface specification. Later, if one of the component developers identifies a flaw in the interface, the people involved can work through how to revise the interface—and the revised specification artifact informs each person how to update their work to match the change. The pattern helps to show how information about a change to the interface specification triggers rework on dependent artifacts.

A good life cycle pattern has procedures to manage the change in artifacts, and how those changes affect other artifacts downstream from them. There are two separate problems these procedures must address:

1. Managing how changes are coordinated across multiple artifacts and through the team while a part of the system is in development; and
2. Ensuring that when a part of the system is complete, all the artifacts are consistent with each other.

Different life cycle patterns approach this in different ways, which we will discuss in later chapters on different patterns. The most common approach is to maintain different versions of an artifact, with at most one version being designated as a baseline or approved version, and other versions designated as works in progress. Many configuration management tools have a way to designate a baseline version, and many software repository tools provide branching and approval mechanisms to track a stable version.

21.8 Life cycle and teams

What is the team size and background? How is it expected to change over time? A small team can often be a little less formal than a large team, because the small team (meaning no more than 5-10 people) can keep everyone informed through less formal communication. A large team is not able to rely on informal communication, so more explicit processes and communication mechanisms are important. Many teams start small when the project is first conceived, but grow large over time. A team that will grow will need to communicate more formally from the beginning than they otherwise might so that as they add people to the team, the larger team works smoothly.

Conversely, if the life cycle patterns indicate that some action will be performed by some person, does the team actually have the staff to do that work? When a project says that some work is to be done and then does not staff that function sufficiently, it sends a message to the team that they should not take the process as written seriously. This undermines the team’s trust. If the function is actually needed, either the team will find an ad hoc workaround or the function will not get done adequately. Either way, there will be a disconnect between what is written down and what actually happens.

21.9 Life cycle and planning

The life cycle patterns are just patterns that provide a guide to work that goes in the project’s plan. The plan is the actual definition of the tasks to be done. When the plan needs to be updated, the patterns provide a template for the work that goes into the plan.

Assembling the plan, however, takes into account many inputs, of which the pattern is only one. Planning involves deciding on the priority and deadlines for work, which is based on project deadlines, risk or uncertainty, and the project’s development methodology.

Chapter 60 discusses in detail how the plan is developed and maintained, including how the life cycle patterns get incorporated.

Consider the following example of how a pattern gets incorporated into the plan. This example shows how the pattern is only a template, and there are many decisions that will depend on other information.

This pattern defines what should happen when a customer requests a change. The basic pattern is that first someone on the team should evaluate the request; this may involve working with the customer to clarify the request, and with other engineers to estimate the scope and cost of the work. The project can then make a decision whether to accept the change or not. If the decision is to make the change, work to build, release, and deploy the update will follow. If not, there is another pattern for how to communicate with the customer that the change will not be made.

The activity starts when the project receives a change request. Based on this, the plan can be updated to include three tasks right away: the evaluation, review, and decision tasks.

At the same time, the planner must make decisions: who should each task be assigned to? What priority should the flow of tasks have? The pattern can indicate the roles involved in the tasks, such as there being a small team responsible for evaluating change requests and a customer representative from the marketing team, but it doesn’t determine which specific people. That’s for the planning and tasking efforts to determine. Similarly, the pattern does not specify how the work should be prioritized relative to other work the same people are doing. The planner incorporates information about how urgent the customer’s request might be and the importance of the customer into the decision. The project might have decided, for example, that there should be a queue of outstanding change requests and they should be evaluated in their order in the queue.

Determining who should be involved in a review of the evaluation might depend on the results of the evaluation. The pattern might indicate that the evaluation should be reviewed by engineers responsible for each high-level component that will be affected by the change. This means that the decision about who specifically will be tasked with the review can’t be made until the evaluation has worked out the scope of the change.

The decision to proceed with making an update will depend in part on whether the team has the time and resources to make the update. The team will need to determine whether adding the work to the plan will cause a problem with meeting deadlines that have been established already, or if it will overload a team that is already busy. This determination will involve analysis of the current plan—something that the life cycle pattern can help with only to the extent that the patterns can help with generating estimates of the work that would be involved.

When the project takes the decision to go ahead with developing an update for the request, the pattern shows that work steps follow to develop a change and then release and deploy the update. When the decision gets made, this will trigger the planning activity to add development and release work into the plan. These are high-level work steps with little detail. The planner will find patterns for these steps and populate those patterns into the plan.

Decisions about the work involved in development will depend on the development methodology that the team has selected to follow. If the update will involved extensive changes and the team is following a spiral-style methodology [Spiral], the development plan might consist of two or three development rounds. Each round would design and implement part of the changes, with a milestone at the end of each round showing how the partial changes have been integrated into the system.

Decisions about the release and deployment work will also incorporate policy decisions about how the team works. Will each change request result in a separate update release? Or will updates be bundled together into releases that combine several updates, perhaps on a schedule defined in advance?

21.10 Principles for a life cycle pattern

In this section I list some principles to consider when designing a workflow pattern.

The act of designing—or refining—a life cycle pattern is an opportunity to think deliberatively about how the team should get its work done. Life cycle patterns are the templates for the project’s plan, and so they should be designed to achieve the work that is needed to move the project forward well.

Designing the patterns ahead of time means having time to define good work patterns. The pattern does not have to be worked out under pressure, as a reaction to something unexpected happening in the project. It can be discussed among multiple team members to get different perspectives and to ensure everyone’s needs are met. Working in advance gives time to check that the steps in the pattern are consistent with each other. It means that there is time to think about what exceptional situations might happen and define what to do in those cases.

Note that if an organization already has an approach to life cycle patterns, whether documented or not, one should aim for continuity with that approach. Anyone already in the organization will know that approach to organizing work; making a major change would mean losing the advantage of established team habits. On the other hand, if the current approach is not working well, then a new project is an opportunity to improve.

The life cycle patterns encode principles and methodology that encourages good work. Principles to consider include:

1. Know the purpose for something before developing it.
2. Build in time for and incentivize deliberative thinking.
3. Assign decision-making authority to an appropriate level based on the nature of the decision.
4. Build in ways to check work, and design them so they are a team norm and not prone to triggering defensive reactions.
5. Build for the longer term.
6. Build in project-wide decision points.
7. Think about exceptions that might happen, how to handle them, and when to change course.
8. Define the work so that everyone on the team can agree when a step has been completed.
9. Give a clear definition for each step of the quality considerations by which the work can be judged.
10. Make the pattern as light-weight as possible without compromising quality.

Chapter 22: Development methodologies

22.1 Purpose

A project should choose life cycle patterns that fit with its development methodology.

A development methodology is the overall style of how a project decides to organize the steps in developing the system. This includes decisions like whether to develop the system in increments of functionality, whether to design everything before building, whether to synchronize everyone’s efforts to a common cycle, and so on. These decisions are reflected in obvious ways in the life cycle patterns a project uses.

There are many methodologies named in the literature: waterfall, spiral, agile, and so on. Different sources interpret each of these differently, and they are rarely compared on a common basis. Some of these, like waterfall methodology, have evolved over time and do not have a single clear source or definition. Others, such as agile development, have a defining document (manifesto) to reference.

All of the methodologies I know of have come to be treated as dogma, and are more often caricatured than treated thoughtfully. This is unfortunate because each of the methodologies has something useful to offer, while all of them are harmful to project effectiveness if taken as dogma or used without thoughtful understanding.

These methodologies can be organized and compared based on a few characteristics.

Rather than try to argue for or against any specific methodology—which is difficult, since most methodologies are hard to pin down among many published variants—I focus on these characteristics, and argue for choosing a methodology that has the characteristics that a project needs.

Size of design-build cycle. Methodologies like waterfall use “big design up front”, where the entire system is specified and designed before implementation begins. Other methodologies break up development into many specify-design-implement cycles.

Size of design-build cycle.

The argument for doing as much design up front as possible is that errors are easier and cheaper to catch and correct before implementation than after. The arguments against are that in some complex systems the design work is exploratory and requires implementing part of the system to learn enough to know how to design—or not design—critical system parts.

Many iterative methodologies claim to be better at supporting adaptation as system purposes change.

Coupled or decoupled design-build. Some iterative methodologies plan to complete adding a feature to the system in one iteration, by executing an entire specify-design-build-integrate cycle for that feature. Other methodologies break up that cycle into multiple steps, and allow those steps to spread across multiple iterations.

Coupled or decoupled design-build.

Advance planning. Some methodologies emphasize planning out work activities as far as possible into the future, while others focus on planning as little as possible in order to adapt as needs change.

Planning to different horizons.

The argument for planning as far as possible into the future is that it gives the team stability: they have a reasonable expectation of what they should be working on now and have a sense of how that work will flow into other tasks soon after.

The argument for planning to shorter horizons is that someone will come along and change priorities or system purpose, and so the work will need to be changed to adapt. Planning too far ahead is wasted effort, it is argued, and gives teams a false sense of stability.

Synchronization across project. Some methodologies that break up development into multiple iterations align all the work being done at one time so that the iterations begin and end together. Other iterative methodologies allow some work iterations to proceed on different timelines from other work.

Synchronized versus unsynchronized tasks.

Synchronizing work iterations across the whole project can provide common points to check that work is proceeding as it should and to share information about progress. However, it can also break up tasks that run far longer than others and result in a perception that the synchronization is wasteful management overhead rather than something useful.

Shared short-term purpose across project. Iterative methodologies can focus the entire team on one set of features across all the work going on at one time, or they can allow different streams of work to have different focuses in the short term.

The argument for this practice is that the more people share a common goal, the more they will be motivated to work together to meet that goal and to defer work that does not address that common goal. The argument for having multiple work streams with different focuses is that too often a project will involve work from different specialties and on different timelines: mechanically assembling an airframe and building a flight control algorithm have little in common.

Shared purpose.

22.2 Methodologies

I present three of the most commonly discussed development methodologies in order to illustrate how they can be characterized. Each of these methodologies has many variants, and all are the subjects of debates comparing tiny details of each variant. The purpose of this section is to illustrate how they can be analyzed, not to capture all nuances of every methodology in use.

22.3 Practical considerations

Most projects actually choose to use a hybrid among the different methodologies. They may start from one of the generally available methodology definitions, but they adapt that template based on the needs of their project and their own experience.

I have been part of several projects that had hard decision points, reflected in project milestones. At these points, the project was expected to provide information that would lead to the project continuing or being canceled: the decision to award the team a contract to build the system, or decisions to continue funding. These decision points impose a degree of waterfall-like structure on the work. For example, one project had to present a proposal to a government agency in order to get a contract to perform detail design and prototype implementation. That proposal involved developing the concept for the system, showing how it met the customer’s objectives, and showing that there was a likely feasible design.

However, once the contract had been awarded, the team used a spiral development methodology to build a sequence of increasingly capable versions of the system, and used those to demonstrate the completion of defined features at the end of each spiral. Within each spiral, the software team used an agile-like approach based on two-week sprints.

Every successful project I have been part of has done some degree of planning and high-level design well ahead of detail design and implementation. When the project used spiral or iterative development, the general flow from one spiral to another provided guidance to keep the work on track to reach the defined end point. When the project used agile methods to manage tasks in the short term, the longer-range plan kept the tasking decisions in each sprint from going off track by providing a basis for prioritizing the work.

No matter what methodology the projects have followed, they have all had some kind of regular cycle for checking in so that project leadership could find out when there were problems, and so that team members could maintain awareness of progress across the whole project. In some cases this looked a lot like agile practice, with short daily stand up meetings and regular, more in-depth discussions. In other cases the nature and schedule for checking in depended on the part of the system people were working on—from continuous interaction in some parts of software development to weekly updates from people doing safety analysis.

All the projects I have worked on have also had tasks varied widely in duration. Many software-related tasks were short, in the range of hours to a few days, while testing or hardware implementation often required weeks to complete. Most of the projects did not try to make all these tasks fit into a synchronized schedule across the project.

In practice, therefore, the projects I have seen that have been successful have applied common sense to the choices they make about how they chose the design methodology for their specific project.

Chapter 23: Example life cycle patterns

23.1 Introduction

In this chapter I survey some of the many different life cycle patterns in use.

The patterns have different scopes. Some cover the whole life of a system, from conception through retirement. Some are concerned only with developing a system. Others focus on more narrow parts of the work.

I group the patterns in this chapter into four sets, based on scope. The first group covers the whole life of a project, without much detail in the individual steps. The second dives into the development process. The third addresses post-development processes—for releasing and deploying a system; these patterns overlap with development processes. The fourth and final group is for patterns with a narrow focus on some specific detail of building a system.

Patterns with different scopes can potentially be combined. Most patterns that cover a system’s whole life, for example, define a “development phase” but do not detail what that is. One of the patterns for developing a system can be used for the details.

Each of the examples will include a comparison against the following baseline pattern for the whole life of a project.

The baseline phases are the same as in Section 21.3:

• Project preparation. Work out and document the processes, rules, standards, and life cycle patterns that the team will use.
• Concept development. Determine what the system needs to be or do in order to be useful to its users and to those who will operate it.
• System development. Design, build, and verify the system.
• Operational acceptance. Review and acceptance by the customer, indicating that the system is ready for operation.
• System production. Build and deploy the system, using the artifacts from system development.
• System operation. Support the system in operation, including fixing flaws and supporting problem resolution.
• System evolution. Add capabilities to the system.
• System retirement. Take the system out of operation and ensure that all artifacts are archived, destroyed, or recycled. Shut down the project.

23.2 Whole project life cycle

These patterns organize the overall flow of a project, from its inception through system retirement and project end. I have selected two examples: the NASA project life cycle, which is used in all NASA projects big and small, and the Rational Unified Process, which arose from a more theoretical understanding of how projects should work.

23.2.1 NASA project life cycle

The NASA life cycle has been refined through usage over several decades. It is defined in a set of NASA Procedural Requirement (NPR) documents. The NASA Space Flight Program and Project Management Requirements document [NPR7120] defines the phases of a NASA project.

The NASA life cycle model is designed to support missions—prototypically, a space flight mission that starts from a concept, builds a spacecraft, and flies the mission.

NASA space flight missions involve several irreversible decisions, and this is reflected in how the phases and decisions are organized. Obtaining Congressional funding for a major mission can take months or years. During development, constructing the physical spacecraft, signing contracts to acquire parts, and allocating time on a launch provider’s schedule are all expensive and time-consuming to reverse. Launching a spacecraft, placing it in a disposal orbit, and deactivating it are all irreversible. These decision points are reflected in where there are divisions between phases, and when there are designated decision points in the life cycle.

There are several life cycle patterns for NASA projects, depending on the specific kind of program or project. I focus on the most general project life cycle [NPR7120, Fig. 2-5, p. 20], which is reproduced below:

The pattern includes seven phases. There is a Key Decision Point (KDP) between phases. Each decision point builds on reviews conducted during the preceding phase, and the project must get approval at each decision point to continue on to the next phase.

The key products for each phase are defined in Chapter 2 of the NPR and in Appendix I [NPR7120, Table I-4, p. 129].

Phase F (Closeout). In this phase, any flight hardware is disposed of (for example, placed in a graveyard orbit or commanded to enter the atmosphere in order to destroy the spacecraft). Data deliverables are recorded and archived; final reviews of the project provide retrospectives and lessons learned.

This pattern of phases grew out of complex space flight missions, where expensive and intricate hardware systems had to be built. These missions often required extensive new technology development. The projects involved building hardware systems that required extensive testing. The NASA procedures for such missions are therefore risk-averse, as is appropriate.

I have observed that many smaller, simpler space flight projects have not followed this sequence of phases as strictly as higher-complexity missions do. Many cubesat missions, where the hardware is relatively simple and more of the system complexity resides either in operations or in software, have blurred the distinctions between phases A through C. In these projects, software development has often begun before the Preliminary Design Review (PDR) in Phase B.

At the same time, I have observed some of these smaller space flight projects failing to develop the initial system concept and requirements adequately before committing to hardware and software designs. This has led to projects that failed to meet the mission needs—in one case, leading to project cancellation.

The phases in the NASA life cycle compares with the baseline model presented earlier as follows.

The NASA life cycle splits the system development activities across four phases. The NASA approach does this because it needs careful control of the design process, in particular so that agency management can make decisions whether to continue with a project or not at reasonable intervals. The NASA approach also places reviews throughout the design and fabrication in order to manage the risk that the system’s components will not integrate properly. Many NASA missions involve spacecraft or aircraft that can only be built once because of the size, complexity, and expense of the final product; this makes it hard to perform early integration testing on parts of the system and places more emphasis on design reviews to catch potential integration problems.

The NASA pattern is notable for some initial work on a mission concept starting before the project is officially signed off and started. There are two reasons for this. First, because all NASA missions have common processes, there is less unique work to do for each individual project. Second, NASA is continuously developing concepts for potential missions, and this exploratory work is generally done by teams that have an ongoing charter to develop mission concepts. For example, the concept for one mission I worked on was developed by the center’s Mission Design Center, which performed the initial studies until the concept was ready for an application for funding.

23.2.2 Unified Process

The Unified Process (UP) was a family of software development processes developed originally by Rational Software, and continued by IBM after they acquired Rational. Several variants followed in later years, each adapting the basic framework for more specific projects.

The UP was an attempt to create a framework for formally defining processes. It defined building blocks used to create a process definition: roles, work products, tasks, disciplines (categories of related tasks), and phases.

The framework led to the creation of tools to help people develop the processes. IBM Rational released Rational Method Composer, which was later renamed IBM Engineering Lifecycle Optimization – Method Composer [ELOMC]. A similar tool was included in the Eclipse Foundation’s process framework, which appears to have been discontinued [EPF]. These tools aimed to help people develop processes and then publish the process documentation in a way that would let people on a team explore the processes.

While the UP and its tools gained a lot of attention, their actual use appears to have been limited. I explored the composer tool in 2014, and found that it remarkably hard to use. It came with a complex set of templates, which were too detailed for project that I was working on. Another author wrote that “RUP became unwieldy and hard to understand and apply successfully due to the large amount of disparate content”, and that it “was often inappropriately instantiated as a waterfall” [Ambler23]. Certainly I found that the presentation and tools encouraged weighty, complex process definitions and that they led the process designer into waterfall development methodology.

The UP defined four phases: inception, elaboration, construction, and transition.

1. Inception. The inception phase concerns defining “what to build”, including identifying key system functionality. It produces the system objectives and a general technical approach for the system.
2. Elaboration. This phase is for defining the general system structure or architecture and the requirements for the system. The results of this phase should allow the customer to validate that the system is likely to meet their objectives. This phase may be short, if the system is well defined and or is an evolution of an existing system. If the system is complex or requires new technology, the elaboration phase may take a longer time.
3. Construction. This involves developing detailed component specifications, then building and testing (verifying) the components. This includes integrating the components together into the whole system and verifying the result. The result is a completed system that is ready to transition to operation. RUP focuses on constructing the system in short iterations.
4. Transition. This phase involves beta testing the system for final validation that the customer(s) agree that the system does what is needed, and deploying or releasing the final software product.

The UP does not directly address supporting production, system operation, or evolution; however, the expectation is that, for software products, there will be a series of regular releases (1.0, 1.1, 1.2, 2.0, …) that provide bug fixes and new features. Each release can follow the same sequence of phases while building on the artifacts developed for the previous release.

The four phases in UP compare with the simple model presented earlier as follows:

The Unified Process provides lessons for defining life cycle patterns: keep the patterns simple, make them accessible to the people who will use them, and put the emphasis on what they are for, not on tools and forms. The basic ideas in UP are good—carefully defining a life cycle, and building tools to help with the definition. I believe that these good ideas got lost because the effort became too focused on elaborate tools and model, losing focus on the purpose of life cycle patterns: to guide the team that actually does the work.

23.3 System development patterns

Some patterns focus only on the core work of developing a system. These patterns generally begin after the project has been started and the system’s purpose and initial concept are worked out. The patterns go up to the point when a system is evaluated for release and deployment. In between, the team has to work out the system’s design, build it, and verify that the implementation does what it is supposed to.

These examples all share the common basic sequence of specifying, designing, implementing, and verifying the system or its parts. Some of the examples include similar sequences of activity to evolve the system after release.

23.3.1 Systems V model

This pattern is used all over in systems engineering work. It is organized around a diagram in the shape of a large V. It is used in many texts on systems engineering; it has also been used to organize standards, such as the ISO 26262 functional safety standard [ISO26262, Part 1, Figure 1].

In general, the left arm of the V is about defining what should be built. The right arm is about integrating and verifying the pieces of the system. Implementation happens in between the two. One follows a path from the upper left, down the left arm, and back up the right side to a completed system.

There is no one V model. There are many variants of the diagram, depending on the message that the author is trying to convey. Here are two variants that one often encounters.

The first variant focuses on the sequence of work for the system as a whole:

The second variant focuses on the hierarchical decomposition of the system into finer and finer components:

The key idea is that specifications, of the system or of a component, are matched by verification steps after that thing has been implemented.

In general this model conflates three ideas that should be kept separate.

1. Development follows a general flow of specification, then design, then implementation, then verification.
2. System development proceeds from the top down: start with the whole system, and recursively break that into components until one reaches something that can be implemented on its own.
3. Development follows a linear sequence from specification and design, through implementation of components, followed by bottom up integration of the components into a system (with verification along the way).

The first two ideas are reasonable. Having a purpose for something before designing and building it is a good idea. There are exceptions, such as when prototyping is needed in order to understand how to tackle design, but even that exception is merely an extension to the general flow. The second idea, of working top down, is necessary because at the beginning of a project one only knows what the system as a whole is supposed to do; working out the details comes next. Again there are exceptions, such as when it becomes clear early on that some components that are available off the shelf are likely useful—but again, that can be treated as an extension of the top down approach.

The third idea works poorly in practice. It is, in fact, an encoding of the waterfall development methodology into the life cycle pattern, and so the V model inherits all the problems that the waterfall methodology has.

In particular, the linear sequence orders work so that the most expensive development risk is pushed as late as possible, when it is the most expensive to find and fix problems. By integrating components bottom up, minor integration problems are discovered first, shortly after the low-level components have been implemented when it is cheapest to fix problems in those low level components. Higher-level integration problems are left until later, when complex assemblies of low-level components have been integrated together. These integration problems tend to be harder to find because the assemblies of components have complex behavior, and more expensive to fix because small changes in some of the components trigger other changes within those assemblies already integrated.

Development methodologies other than waterfall address these issues better, as I discussed in Chapter 22.

23.3.2 Systems or software development life cycle (SDLC)

There are several life cycle definitions for system development, primarily of software systems, that go by the SDLC name. They generally have similar content, with variations that do not change the overall approach.

I have not found definitive sources for any SDLC variants. It appears to be referenced as community lore in many web pages and articles.

The core of the SDLC consists of between six and ten phases, depending on the source, that give a sequence for how work should proceed in a project. The phases are:

• Initiation(*): Identify a need or opportunity for a system.
• Concept development(*): Work out the system’s purpose and scope, along with a rough concept of how it might work. Do feasibility and cost-benefit analyses. Some variants merge this phase with the next two.
• Plan: Develop how the project will operate, including management plans, procedures, and resource estimates.
• Requirements(*): Develop requirements (specifications) for the system based on customer needs developed in earlier phases.
• Design: Develop a design for the system that will match requirements identified in the Plan phase. Includes making build-versus-buy decisions. Some SDLC variants separate this into Preliminary Design and Detailed Design phases.
• Implement: Write software that follows the designs from the previous phase, or acquire it from outside sources. Some variants separate this into Implementation and Unit Test as one phase and Integration as a second phase.
• Test: Perform testing on the software to verify that it meets the system objectives and requirements. Some variants mention review and analysis of the implementation artifacts as part of this phase.
• Deploy: Release or manufacture the system artifacts and put them into operation.
• Maintain: Fix problems identified in operation, and develop new features to extend the system.
• Dispose(*): Take the system out of service, dispose of any physical assets, and potentially preserve any data generated by or stored in the system.

Phases marked (*) are not included in all sources.

Most discussions of SDLC stress that the pattern is meant to help organize a project’s work, not to dictate the sequence of activities. Some sources then discuss how the SDLC relates to development methodologies. A project using the waterfall methodology would perform the phases in sequence. Iterative and spiral development would lead to a project repeating parts the SDLC sequence multiple times, once for each increment of functionality that the project adds to a growing system. A project using an agile methodology would perform tasks at multiple points in the SDLC sequence in any given iteration, as long as for any one part or function of the system the work follows that sequence. I discussed how life cycles fit with development methodologies in Chapter 22.

23.4 Post-development patterns

23.4.1 EVT/DVT/PVT

Many electronics development organizations use a set of development and testing phases:

• Engineering Validation and Testing (EVT)
• Design Validation and Testing (DVT)
• Production Validation and Testing (PVT)

This set of phases is intended for developing an electronic hardware component, such as an electronics board. Developing this kind of hardware differs from developing a software component: while software source code can be compiled and tested immediately, a board design must be built into a physical instance before much of its testing can happen. Simulating the board can be done earlier, of course, but much testing is only done on the physical instance. This is especially true for integrating multiple boards together.

This pattern also addresses not just the design and testing of the component itself, but also the ability to manufacture it—especially when the component is to be manufactured in large numbers. The NASA, V, and SDLC patterns do not address manufacturing specifically; this pattern can be combined with those if a project involves manufacturing.

23.5 Detail patterns

The last two patterns have to do with managing changes to the system: when errors are found, and when customer needs change.

Both these patterns apply to specific, short parts of a project. They apply as needed—when a error report or a change request arrive. Both also potentially involve repeating parts of the overall development life cycle pattern. Both may be used many times in the course of a project.

23.5.1 Defect or error management

This life cycle applies when someone reports a defect or error in the system. It includes fixing the problem and learning from it.

Common practice is to use an issue or defect tracking tool to keep track of these reports and the status of fixing them. Many of those tools have an internal workflow, and parts of this life cycle pattern end up embedded in that internal workflow.

There are two different times when people handle error reports: when errors are found during testing, before an implementation is considered as being verified, and later, when a verified design or implementation must be re-opened. In the first case, the people doing verification are expected to be working closely with the people implementing that part of the system; the pattern for that activity amounts to reporting an error, fixing it, and verifying the fix.

The general pattern for addressing later errors is:

1. Reporting. Someone finds an error and reports it into the tracking tool.
2. Triage and ranking. Determine what to do about the report. Someone investigates the report to determine whether it is understandable and actionable. This may involve communicating with whoever reported the problem to get more information, either about their expectation or about what they found. The result is either accepting or rejecting the report. If accepted, the report is given a priority level (typically one of four or five levels) and a likely part of the system affected. If rejected, the result is an explanation of why the report will not be acted on.
3. Assignment. Who will be responsible for resolving this error? Most projects have a standard procedure: whoever is responsible for the component identified as the likely source, or people can pick up reports when they have time, or a manager can make assignments.
4. Analysis. What is the actual problem that caused the report, and how can a fix be verified? This investigation might involve working to reproduce the problem. The analysis may find that the source of the erroneous behavior is a defect in a different part of the system than originally determined during triage, which may lead to assigning the responsibility to someone else. The analysis may show that the problem is broadly systemic or that it arises from multiple defects in multiple parts, in which case several people will be involved with fixing the problems and overall responsibility for the report may be moved to someone who can oversee all the affected parts.
5. Fix. Making changes to the system amounts to recapitulating the overall life cycle pattern for building a part of the system. This can be seen as an instance of rewinding progress, as discussed in Section 20.3. The fix might be simple—there is one part of one component that is reimplemented, a test is developed to check the change, and it can be reviewed and approved. On the other extreme, the problem might come from a high-level design decision; the fix may involve changing that design, which in turn changes the specifications for multiple components, each of which must have their designs updated, their implementation and tests updated to match.
6. Verify. Once a fix has been implemented, the changes are verified. The fix is verified to ensure that it actually addresses the reported problem, and that it hasn’t created new problems. The changes may have affected how components integrate together, in which case the verification status of integration and interactions among them may be invalidated by the change, and the integration must be revalidated.
7. Review and accept. Once the fixes are complete and have been verified, the fix can be reviewed and accepted. At this point the updated designs and implementations are baselined (that is, made as the current mainline working version). The record of the error can be marked as completed.
8. Learning. Is there something that can be learned from the error and its fix that can be used to avoid similar errors in the future? This may be informal learning by the people involved, or it may be important enough to document and used to educate others on the team.

23.5.2 Change requests

From time to time, someone will request changes to the system. The request may come from a customer, asking for a change in behavior or capability. The request may come from the organization or funder, reflecting a desire to meet a different business objective. The request might even come from a regulator, when the regulations governing a system change or when the regulator finds a problem when reviewing the system.

The pattern for handling a change request has much in common with the one for handling a defect report.

After receiving a request, someone evaluates the request to ensure that it is complete and that they understand the request. After that, there is a decision whether to proceed making the change and, if so, what priority to put on the change. After making the decision to proceed, there are steps to design, implement, and verify the changes and eventually release the new version of the system.

Change requests differ from defect reports in two ways. First, requests for changes do not reflect an error in the system as it stands. The team can proceed building the system to meet its current purpose and defer making changes until after the current version is complete and released. Second, most requests are expressed as a change in the system’s purpose or high-level concept rather than as a report that a specific behavior in a specific part of the system does not meet its specification or purpose. A high-level request will have to be translated into, first, changes in the top-level system specification, and then propagated downward through component specifications and designs to work out how to realize the changes. This sequence of activities to work from the change of objective to specifications to designs to implementations is essentially the same as the activities to specify, design, and implement the system in the first place. In the pattern shown above, the “develop update” step amounts to recapitulating the overall system development pattern.

The decision to proceed with a change or to reject it depends on whether the change is technically feasible and whether it can be done with time and resources available. This depends on having an analysis of the complexity involved in making a change. Ideally, the team will be able to estimate the complexity with reasonable accuracy and little effort. Analyzing a change request will go faster and quicker if the team has maintained specification and design artifacts that allow someone to trace from a system purpose, down through system concept, into specifications and designs, to find all the parts of the system that might be affected by a change. If the team has not maintained this information, someone will have to work out these relationships from the information that is available—which is difficult and error-prone.

23.6 Comparisons and lessons learned

The life cycle patterns in this chapter have all been developed in order to guide teams through their work. To meet this objective, they have to be accessible and understandable by the teams using them; they can’t be explained in legalistic documents that include many layers of qualification and exceptions. Some of these have passed this test and have been used successfully. Others, such as the Unified Process, have not caught on.

Some of the patterns cover the whole project, while others address specific phases or activities. One pattern often references other patterns: for example, a high-level pattern like the NASA project life cycle uses lower-level patterns for developing components or handling change requests. Some low-level patterns, such as handling change requests or error reports, can end up using or recapitulating higher-level patterns.

The specific patterns that a project uses depend on that project’s needs. A software project that is expected to be continuously reactive to new customer needs works differently from a project that is building an aircraft, where rebuilding the airframe can cost lots of money and time. The NASA approach is influenced by the US Government fiscal appropriation and acquisition mechanisms, which require programs to have multiple points where the government can assess progress and choose to continue or cancel a program.

All of these patterns implicitly start with working out the purpose of some activities before proceeding to do detailed work.

These patterns also implicitly reflect the cost of making and reversing a decision (Section 21.10). The NASA life cycle puts design effort before a decision to spend money and effort building hardware. The change request and defect report patterns place evaluating the work involved ahead of committing to make a change.

Part VI: Reference life cycle

A reference life cycle pattern for projects. It models what a full life cycle contains, and can be the basis for developing an actual project’s life cycle.

Chapter 24: Introduction

The previous chapters have introduced the ideas of life cycle patterns and development methodologies, along with the ways that the two affect each other. Chapter 22 introduced a number of characteristics that one can choose to match a project. Chapter 23 presented a number of example life cycle patterns, along with a rough framework for comparing the examples.

In this chapter, I present a reference model of development methodology and life cycle patterns. This approach is based on the approaches I have used myself or have observed others using in successful projects, along with learning from projects that have gone poorly. These recommendations do not attempt to follow any of the development methodologies dogmatically, instead taking the parts from several of them that work well. In other words, I have tried to distill a pragmatic set of solutions from the many options available.

The reference life cycle covers the entire life of a systems-building project. It has four high-level phases: preparation, development, operation, and ending.

Project preparation is about setting up the project: how it will work, who is sponsoring it, who is funding it. Development covers working out what the system is for and then designing and building it, until it is ready for use. Operation is about producing the system, deploying it, using it, and evolving it. Ending is about shutting down the project when its work is done.

This reference also includes a project support “phase”, which includes all the activities that go on throughout the project to support operations.

Some projects are only concerned with building a system; once the system has been implemented and tested, it goes into production or operation and is no longer the concern of the development team. Those projects skip the operations phase. Most projects, on the other hand, have some level of involvement after the system is deployed and in operation, such as fixing bugs or enhancing the system. These projects involve all the phases.

The phases in the top-level life cycle in turn expand into more detailed patterns. Development consists of working out a purpose and a concept for the system, then developing a system to match, ending with a review to determine that the system is acceptable for putting into operation. Operation expands into a pattern of several phases, which I will discuss below.

Some projects will spend most of their time in development, while others spend most of their time in evolution after the system is in operation. Exploratory spacecraft missions usually consist mostly of development, since once the spacecraft is launched there is little opportunity to change the spacecraft beyond the occasional software update. Mass-market consumer software, on the other hand, often spends as little time as possible on initial development and can spend years developing upgraded versions to keep consumers satisfied. This reference life cycle fits both kinds of projects.

The arrows in this diagram show how information and artifacts flow from one phase to another, but they do not necessarily indicate complete temporal orderings. For example, the project preparation phase often lasts quite a while, and overlaps early parts of the development phase. Within operation, different customers might deploy and operate their own instances of the system, and the project may be working on multiple system improvements at once.

Two of these phases—system development and system evolution—involve designing and implementing parts of the system. These are the two phases where a development methodology applies.

I will discuss each of the top-level project phases in turn in the coming chapters.

24.1 Projects with proposals

Some projects require developing a proposal to get funding or approval to proceed.

The life cycle for this kind of project adds a phase between preparation and development to develop a proposal. Developing the proposal typically involves developing the purpose and a preliminary concept for the system, so that the potential customer or funder can understand what they will getting if they agree to fund developing the system. The initial concept is then documented as part of the proposal itself, which is typically a document (often a large document) explaining what the system will be, how it responds to the customer’s requirements, how long it will take to develop, and how much it will cost.

Much has been written about how to do proposal development well. There is best practice for how to organize a proposal development team and what kinds of reviews are helpful.[1]

After the customer or funder has agreed to the proposal, system development proceeds as it does for other kinds of projects.

24.2 Project-wide decisions and reviews

Projects have times when there will be a decision whether to continue the project, end it, or continue with significant changes. Some examples: whether to start a project, when additional funding is needed to continue, or at periodic progress reviews.

These are often not driven by progress on making the system. They can be driven by external considerations, such as the need for funding, or by a regular cadence of progress checks.

Such reviews or decision points do not fit neatly into the flow of phases defined in the life cycle pattern. When multiple steps are in progress concurrently, as happens during most of the development phase, the decision often happens in the middle of several of them. Preliminary specification or design reviews are also common; they happen part way through specifying or designing part of the system. Design reviews often mean that design should have reached a given level of completeness for the top X layers of components in the system.

I will note some representative decision points in this reference lifecycle, but the actual milestones are project-specific.

Chapter 25: Project preparation

The project preparation is for getting together the things that the team will need to operate.

Chapter 26: Project support

Project support covers all the various things done continuously in the project to keep the team working. Project support starts with the beginning of the project and ends when the project ends.

This phase includes work to monitor and manage parts of the project. Teams are one example (Section 19.3.1); maintaining plans and tasking (Section 20.5) is another. Tracking project risk (Chapter 62) and technical uncertainty (Chapter 61) supports planning.

Other elements of project support include:

• Artifacts: setting up and maintain the tools that store artifacts and handle artifact-related workflows.
• Tools: defining how to acquire and maintain tools, and how people will be trained to use them.
• Teams: hiring, on boarding, and off boarding procedures; how to maintain a team directory (Section 19.4).
• Space: the office and work space needed for the team.
• Partners: how to maintain relationships with partner organizations, such as subcontractors.
• Finance: tracking budgets, working with accounting organizations.
• Production and vendors: how to select and work with vendors.
• Relations: who will be responsible for maintaining relationships with customers, the organization, funders, and regulators (Section 16.2).

These efforts, similar to project preparation, will usually start small and develop over time. Similar principles apply to the timing of work on project support.

Chapter 27: Development

The development phase sees the project work out what the system is supposed to do, and then build the system to meet that objective.

27.1 What is development?

Before going into the sub-phases that make up the development phase in the large, it’s worth thinking about how a system actually gets developed. A great many systems have been built over the centuries without the benefit of methodologies; with some experience, good systems engineers usually have intuition that guides them through development.

Development starts with a rough idea of what the system is for: what problem the system will solve, or what it can do for people. An aqueduct begins with the idea that something should transport water from a source to a town. A pump driven by a steam engine starts with an idea that a machine could pump water out of a mine better than a human- or animal-driven pump, and thus allow mines to go deeper than they had before.

The people thinking about the problem to be solved also often have some approaches in mind that might be applied. Someone responsible for moving water into a town might know about aqueducts that have already been built. Steam pumps were developed incrementally by many people over a period of over two hundred years.

For developing modern complex systems, the development process still begins with a general idea of what the system might do and what problems it might solve, perhaps with some key technical approach in mind.

The team needs to get from this general idea to a clear and precise definition of what they need to design and implement. This does not occur in one step; the detailed design of the system does not spring fully-formed from the chief engineer’s head. Instead, the team starts with a vague understanding and refines it bit by bit until it is clear enough for design and implementation to start.

The team does need to understand the system’s purpose before working out how the system should work. However, in practice these are often parallel efforts, where some people work with customers and other stakeholders to clarify the system purpose while some people begin to brainstorm ideas of what kind of system might meet that purpose. As the understanding of purpose becomes clearer, those who are investigating what the system might look like—the concept of the system—will refine their ideas. Those who are working on the system concept track updates to the purpose, often feeding questions back to stakeholders when they find something potentially ambiguous or when they suspect that some part of the purpose might not yet be worked out.

The system concept represents the bridge between understanding the customer’s needs and building the details of the system. The concept sets the general approach that the team will use. Working out the concept is a time for creativity, when the team can entertain many possible ways to build the system, eliminating those that aren’t likely and refining those that are promising. The team evaluates these possible approaches along the way to see if they are likely feasible to build and to meet the system purpose.

The team may be tempted to turn the concept-building exercise into a full system design exercise. This is unwise. First, the techniques used to develop a system concept are meant to be fast and fluid, not working to the degree of rigor that design and implementation require. Second, this can lead to a concept and design period that drags on and on when the team needs instead to make a decision about the high-level structure and then move on to investigating design based on that decision. Third, stopping to review the basic concept before committing to it makes for a better concept that will better guide the team later.

This means that a system concept will be (and should be) incomplete. It should show some of the big ideas of the system’s structure, and it should show that these ideas are likely to meet the stakeholders’ needs and are likely to be technically feasible. It should be accurate, in that anything named in the concept should in fact be a necessary part of the system, but it should not be precise, having all of the details worked out.

Once the team has a concept, it is a good time to step back. Is this system still worth building? Is it likely to be feasible? Is it going to be a good answer to customer needs? And is it plausible that the resources needed will be available?

As the development work moves forward, the team will refine the concept. They will find things missing in the concept and have to find designs that fill those gaps. They will find inconsistencies or mistakes, and they will have to correct them. At the same time, customer needs may change—so the initial concept will always be different from the final system.

The level of detail and analysis needed in the concept depends on the project. A project that is building a revolutionary system for potential future customers probably only needs a rough sketch of the system, since investigations will continue for months or years into what those customers really need. On the other hand, a project that is answering a request for proposal typically needs a much more developed concept in order to explain to a funder what they will get and why their funding will be a reasonable risk.

Once the purpose and concept are completed, the team can turn to actually developing the system itself. In practice this is rarely a sharp transition; instead, some part of the team may begin moving forward in working out a system specification even before the concept is finalized, or they may begin prototyping parts of the system that seem especially uncertain.

27.2 The development phase

Development consists of many sub-phases. Purpose development comes first, in which the team determines what customer needs the system will address. After determining the system’s purpose, the team develops a high-level concept for the system, then builds the system itself. The development phase ends when there is agreement that the built system is ready to be produced, deployed, and put into operation.

The first two steps set the direction for the system development work. Purpose development establishes a record of who the stakeholders are for a project, and what each of them needs the system to do. This record of the system’s purpose will be incomplete, initially, but it must be accurate at the time it is documented. The concept phase then provides the time to explore different ways that a system might be built to meet those needs. The concept records a high-level picture of how the system will behave, the environment in which it operates, and some of the main top-level components that will make up the system. The concept phase is also the time when constraints related to security and safety are refined, turning general objectives coming from the customer or other stakeholders into more precise statements of what those objectives mean. Part way through or at the end of concept development is a good time for a review and decision about whether to continue the project.

The system development step in turn consists of many tasks. In this reference approach, the development phase is organized first into a number of system feature development phases, using the development methodology to determine what those phases are. Each system feature development phase, in turn, is organized as a sequence of specify-design-implement-verify patterns.

In this section, I will first discuss the development phase as a whole, then go into more detail about each of the subphases and development methodology.

27.3 Purpose development

The purpose development phase is for working out in detail what the system is to be, in terms of what it will do for its users and who those users are (Chapter 9).

The people responsible for working out the purpose work with the customer (or a proxy for customers, when the customers are hypothetical; see Section 16.2.1). This requires the team to work directly with the customers, in order to understand not just what the customers are saying they need but also to identify implicit needs and to find constraints on the system that the customers may not be able to articulate.

The team does similar work with other stakeholders. They identify the objectives that their organization has: is it to make a certain level of profit? Are there time constraints on demonstrating capability? Who might be the funder, and what are they looking for? And finally, who might have regulatory authority over the system, and what regulation or standards apply? All this information creates constraints on how the system can be built and what it can do, and will be considered when determining whether these other stakeholders will agree for the project to continue.

The needs found in this phase define objectives that the system should try to address. The constraints, on the other hand, define things that must be true about the system. ! Unknown link ref

I discuss working out system purpose further in Chapter 31.

27.4 Concept development

The concept development phase is the transition between working out system purpose and beginning to design the system in detail. It is a time to work out an initial, rough idea of how a system might be built to meet the purpose and constraints worked out in the purpose development phase. It is a time to brainstorm many different possible approaches and to be creative. These different approaches can be evaluated and narrowed down to one concept. That concept is the start, not the end, of design; it will guide the work in the subsequent system development phase.

The system concept is a sketch of the system on paper or similar media. It should cover all the major behaviors of the system, but it should not go into great detail about how those will be achieved.

The concept has two general parts: an external view and an internal view. The external view takes a black-box perspective of the system, and includes:

• The users and external interfaces of the system, documenting who will use the system;
• The externally-visible behaviors; and
• The environments in which the system will operate.

The internal view is an initial sketch of the insides of the system’s black box. This view includes:

• The first-level parts or components of the system, and how they are connected; and
• Behaviors that these components should have.

The concept does not usually go more than one or two levels deep in the component breakdown.

This information can be recorded in different forms, and it usually takes more than one to capture it adequately.

• Key use cases capture information about users and expected behaviors.
• A structure diagram, showing the major components and their relations, captures some internal information.
• A concept of operations provides a narrative of how the system will operate and illustrates both internal and external behaviors.
• A requirements matrix or requirements tracing tree shows how the system concept meets the system purpose.

Documents recording analyses complement these records. The whole collection of concept documents also records the rationale for decisions taken, and perhaps includes records of alternative designs that were considered and not chosen.

The concept is used for three purposes. First, it reveals whether there is likely to be a feasible approach for implementing a system that meets the customer needs. Second, it provides an illustration to customers and other stakeholders that they can use to validate whether the concept meets what they expect their needs to be. Third, it provides a guide as the team begins to specify and design parts of the system for real.

A likely-feasible concept is one where there is likely to be some way to design and implement each of the high-level parts of the system, and that combining those parts will likely satisfy stakeholder needs. The concept can only be likely, because it is supposed to be developed quickly; the uncertainties about whether the concept will actually work are not completely resolved until the whole system has been built and verified. The process of developing the concept can generate a list of what technical uncertainties people have found or suspect. (These uncertainties guide work planning as the project moves into the system development phase.)

The concept gets reviewed by stakeholders, including customers or customer proxies. While a stakeholder might look at the list of their needs as generated in the previous phase and think it complete, I have found that when they step through how a system concept will operate they get a different perspective and come to realize things they missed in the list of needs. When they find that a system concept appears to meet all their needs, the act of validating the concept with them provides them confidence in the project.

Finally, the road that the team follows from initial idea to a complete design (and implementation) has to start somewhere. The concept provides that starting point. The high-level components identified in the system concept become the starting point to specify, design, and build all of the rest of the component in the system.

Chapter 32 discusses the work involved in making and documenting the concept. To summarize that chapter, developing a system concept involves brainstorming many possible approaches to meeting customer needs and sketching them out. These different approaches get evaluated and compared to find out how well they meet the system purpose and how feasible they are; this often involves doing simple analyses. The evaluations show where there are gaps in meeting customer needs or in the technical solution. The best possibilities get refined or combined and improved until the approaches have been narrowed to one best option.

I have said that the concept should be likely feasible, and that the technical uncertainty and project risk uncovered in the investigation should be acceptable. The obvious next question is, how likely or how much uncertainty? In fact these uncertainties and risks are not generally quantifiable, as they deal in unknowns and the point of the concept development exercise is to expose unknowns and not to work them out. Qualitatively, some projects can accept more risk than others: a startup that is developing a speculative new technology can accept far more risk than a project proposing a system for a fixed-price contract. The decision will require a judgment call on the part of project leaders.

27.5 Development methodology

The system development phase is about creating the system based on the concept worked out in the previous phase. At the end, the project has the artifacts for a working system ready to hand off to production and deployment. Along the way, the project may need to meet other milestones—preliminary and critical design reviews for government customers, or feature demonstrations for funders.

The reference development methodology structures how the team does the work to design, implement, and verify that system. It is based on the spiral or incremental methodology. Project leadership works out a set of intermediate milestones where the team builds and demonstrates some set of system features working—usually integrating different parts of the system along the way. There is a life cycle phase leading up to each of these milestones, in which the team does the tasks needed to add features to the system. These are called feature development phases. Each feature development phase has an expected duration. If it appears not to be on track to meet that deadline, the team takes this as a signal that corrective action is needed. Unlike in the spiral methodology, this methodology leads to multiple overlapping feature development phases, running in parallel on different timelines and working toward different milestones.

This approach was motivated by several goals.

1. Provide mesoscale guidance to the development process, so that there is continuity and planning while providing a way to adapt when needed. This level of guidance operates on a time scale longer than a few days or a couple weeks, but less than the whole project; it is intermediate between the time scales used in agile and waterfall development.
2. Allow threads of activity (feature development phases) to operate on longer or shorter timelines as is appropriate to the work.
3. Allow different feature development phases to follow the microscale methodology appropriate to the work in that phase. For example, software development often benefits from short, agile-like sprints within one feature development phase, while fabricating large mechanical structures is better done using techniques derived from construction industries.
4. Promote interaction and collaboration across parts of the system during design, while avoiding involving people working on unrelated parts of the system. Those who are actively working together to build some interconnected capabilities in a few components need to communicate frequently, but people working on some other part of the system do not need to sit through meetings discussing things they aren’t working on.
5. Support integration-first and uncertainty-first planning practices (Chapters 46 and 61).
6. Support a partial plan—one that is concrete in the short term and less so in the medium to longer term, with growing specificity as the project progresses (Chapter 63).

Compare this approach to waterfall and agile development methodologies.

Waterfall development, practiced strictly, does not handle uncertainty or adaptation well: the system is designed up front, and implementation follows thereafter. In practice, projects nominally using the waterfall methodology often develop intermediate milestones to organize the work.

Agile development, on the other hand, can lead teams to constantly change direction—unless they develop a plan with some longer-term objectives. When they do so, agile development ends up looking a lot like this reference methodology. Short sprint periods can also work poorly for parts of a project doing work that does not complete within one sprint, like building an airframe or developing detailed analyses.

Example. Consider the following example, taken and simplified from a spacecraft project I worked on. The mission involved multiple spacecraft working together to perform a science mission.

The mission’s concept development defined the overall design of the system: multiple spacecraft, communication links between them, communication with ground stations, and so on. The concept also defined an initial breakdown of the system, where the spacecraft had a set of major subsystems like structure, power, avionics, sensors, flight software, and so on. The concept identified some existing software and hardware designs that could be re-used for this mission.

The development phase, then, was about building hardware, software, and operational procedures that would implement that concept.

The team worked out the major steps that had to happen to build the system, such as designing the avionics, designing the structure, testing and integrating them, and putting sample spacecraft units through environmental testing (heat, vacuum, vibration). The project also would build software to run each spacecraft, which involved tasks like prototyping algorithms for attitude control and then verifying that they would work in testbed equipment. These major steps were partly worked out based on experience on previous missions, and partly from working backwards from the high-level system design to determine major functions to be implemented.

The following shows the first part of the sequence of feature development phases for the main flight software (simplified and abstracted from the original). The flight software had a series of milestones that started with the basic software infrastructure and a simulation environment for testing it. Later milestones then added capabilities one after another. Each milestone integrated new functions across several different components. In most milestones, the work involved behind the scenes was as important as what was overtly demonstrated; for example, the first demo was as much about establishing a software configuration management and build system as it was about demonstrating simple software running.

This project made extensive use of software skeletons or scaffolds, mockups, and emulations. This is typical of a project that prioritizes integration over feature depth. In this case, the main spacecraft control software for the first couple of demos was a simple skeleton of what it would become. The software modules involved could start up, and interact with some others in simple ways, but there was no real logic in the control. Building this part first reduced integration risks that the control software modules would not interact properly with the middleware and operating system on which they ran—and indeed showed middleware bugs that cause the system to crash. By the third demo, the team added basic attitude control logic to the control software. This attitude control still only had limited function; its purpose was as much to show that the control software could interact with (emulated) sensors and actuators.

27.6 System feature development

A system feature development phase is a stream of work that adds a defined set of features (the purpose of the phase) to the system, ending in a milestone with those features implemented, integrated, and demonstrable. It starts with design work that has already been done and the purpose of the work, and ends with system artifacts updated to meet the phase’s purpose.

This approach to organizing development is focused on the features rather than on the components or component breakdown. One feature development phase usually involves several components (and their subcomponents). It promotes the integration of work across parts of the system.

Reference pattern for feature development. A feature development phase recapitulates the life cycle of the overall system development life cycle. It starts with purpose, works out a concept, then proceeds into the specification, design, implementation, and verification of parts of the system to build in that purpose.

The concept for a feature development phase includes working the general design approach for adding the phase’s features. As with the system concept, the feature concept involves brainstorming different ways to implement the features, along with evaluations of the alternatives until the team selects one concept. The concept for the features should give a general idea of what components will be modified or created in this phase, along with the internal structure among those components and a narrative of how they will interact (the concept of operations for the features).

Identifying the components that will be affected is key to being able to scope how much effort will be required to implement the features, and who will need to be involved in the work.

The next step is to develop or modify specifications for the components involved (Chapter 33). These detail how the components are to behave and the non-functional attributes they are to provide. This may involve adding to or modifying the top level system specifications, or flowing those specifications down to components. Security, safety, and reliability specifications are particularly important.

Design follows specifications, working out how each component can be built to provide the behaviors and properties it is specified to have (Chapter 37). Design may require evaluating alternatives, perhaps by modeling or prototyping (Section 8.3.5; Chapter 41).

Two separate and independent implementation steps follow. One step implements components and changes to components, following the design. The other step works out how to verify the features in the feature development phase, including verifying both the individual components by themselves (using unit tests, for example) and the features that are provided by the components integrated into the system. If the verification implementation runs ahead of the component implementation, the component implementers can verify as they go (using test-driven development).

As parts of the feature set are implemented, they are verified. By the end of the feature development phase, the components created or changed in the phase and the features the phase is adding are all verified.

The feature development phase ends when the team successfully demonstrates that the system now has the features they have worked to implement. This demonstration might amount to showing that the new system version has passed its verification checks, but doing an actual demonstration gives the people who did the work an opportunity to show the rest of the project what they have done and for the project as a whole to celebrate their work.

Once again, note that this work is organized around the features, not the components. This methodology does not necessarily mean implementing each component’s changes in isolation, verifying those, and then verifying their integration. Rather, the team can order the work however works best for the particular task at hand. For example, an integration-first approach might lead the team to build simple skeletons or mockups of component changes and focus on checking out how the components will interact before implementing detailed changes to the components—which means verifying integration before verifying the unit components. (Of course, the finished changes still need to be verified as a whole before the verification work is done.)

The reference pattern for the feature development phase, in the diagram above, includes review milestones for each of the steps (concept, specification, design, implementation, verification) involved. These reviews serve two purposes. First, they are an opportunity for someone independent to check the work in order to find things the team doing the work might miss. Second, they provide an opportunity for the team working on the features to pause long enough to ensure that they all understand the work in the same way.

Finally, the team responsible for a feature development phase may decide that the phase is large enough that it should be split up into subphases. Each of the subphases might have its own milestone goals; those subphase goals build on each other to reach the features of the main feature development phase. These subphases might focus on individual components or smaller groups of components, or they might split the work into sequential steps, or some combination of the two. These subphases follow the same pattern as the higher-level feature development phase of which they are part.

27.7 Recursion to component development

A system feature, in the end, is made up of behaviors and properties of a number of components. That is, system features are emergent from the individual components involved.

The work to implement a system feature is thus made up of the work on each of the components, along with the effort to integrate those components and their changes. The team working out the concept for the feature determines how parts of the high-level feature are allocated to components. That is, they work out what behaviors or properties are needed from each component so that together they produce the high-level feature. Along the way during concept development, the team works out what components are affected by the feature development work.

The feature development life cycle pattern for the high-level feature applies for developing the changes to each of the affected components. Just as the feature as a whole has concept, specification, design, and implementation steps, so do each of the components. Developing the concept for the feature includes developing a concept for each affected component. Developing the specification for the feature leads to developing specifications for each component, and so on. The implementation of the feature is the implementation step for each component.

The people who are working on all these component pieces coordinate their work so that it all integrates properly and produces the desired features.

That coordination means that the work on each component moves at a pace at least partly constrained by the work on other components: for example, the specification step for any one component cannot be completely finished until the specifications for all the affected components are finished. Otherwise, the specification work in some other component could reveal a surprise that affected the specification that was thought to be finished.

At the same time, teams rarely just stop and sit idle when the work on some component lags. They proceed from specification to design to starting implementation, accepting the risk that some surprise may happen that will require them to re-do some amount of work. The choice of how much work to do at risk has to be made based on the usual estimates of likelihood and consequence. If the work on some other component is almost done and is in the final stages of cleaning up details, the likelihood of finding something that will require a change to other components is unlikely. On the other hand, if the work on some other component is just getting started, then the chances of a surprise are high. If part of the component in question appears to be fairly immune to changes in other components, then there is little risk of having to redo that work. For example, if the component will definitely need to communicate over a network with other components, then getting network communication designed is low risk.

The figure above illustrates how the work for a feature is coordinated across all the components. The top row shows the steps or phases for the feature as a whole. that work is broken down into the work for two components, shown in the middle two rows. The components each follow the feature development pattern of concept, specification, design, implementation, and verification. The last row covers the thread of work done to address integrating the changes to individual components, and it follows a reduced form of that pattern. The feature integration thread of work is primarily about checking that the work on the components properly combines to produce the high-level system features, and so it focuses on verification methods for this integration.

The figure also shows that the concept development work for the high-level feature and the affected components may often be done as a single task. If the feature and components are simple enough, a small group can work out the concept together and produce one set of concept artifacts that cover both the feature as a whole and its effects on specific components. In this case, the artifacts for each component will reference the shared concept artifacts; after a while, the records for a component may reference several concepts for different features.

If the feature or the components are more complex, the work may need to be divided up so people can work on different parts in parallel, combining and reconciling the pieces before the concept is completed. The artifacts for the components will then reference their own concept for that feature as well as the high-level feature concept documents.

27.8 Feature development variations

The feature development pattern in the last section covers the simplest case: when the team is designing and building a straightforward feature. There are three variants to consider: when the component carries enough uncertainty that prototyping is warranted; when the component will be acquired from outside the project rather than built in house; and the specific needs for implementing hardware components.

Prototyping. Prototyping is used when there are possible technical approaches to designing some part of the system, and the technical uncertainty is too high. In these cases, taking steps to reduce the uncertainty before committing to one particular design can lead to better outcomes.

The uncertainty can take different forms. In one case, the team might have an idea, but they don’t know if it will work correctly. In another case, they may not have an idea for a solution, and they need to explore and learn in order to find possible solutions. Or the team might have a solution, but lack skills essential to completing design or implementing it. Finally, the team may have a solution that is not technically mature enough, and they need to validate its suitability. In each case, developing a prototype of some kind can help.

The prototyping effort is added to the design step. The prototype might take the form of a simple implementation, or of a model of a possible solution. Any prototyping effort should have a clear purpose: to see if an idea works (and working out what it means “to work”) or the like. The focus must be on learning what is needed as quickly as possible. The work should prioritize speed of learning over quality of the prototype implementation.

Prototyping can be a necessary part of learning about a design and managing its uncertainty, but its contribution to the system is indirect—by leading to a good design. The amount of effort or time spent on the prototype should be bounded so that the prototyping effort does not take over the development effort.

The principles about prototyping (Section 8.3.5) apply. The prototype artifacts should be built as quickly as possible to maximize efficient learning, without putting in effort to make them high quality. The artifacts that come out of the prototyping work must not end up in the real implementation.

Acquired component. Sometimes components are best acquired from somewhere else rather than being designed and built by the team. This might involve reusing a component from another project, or using an open source design, or purchasing a component from a supplier. Acquiring a design or component can take advantage of work that others have already done, reducing development costs. It can take advantage of expertise that the team does not have itself, such as a supplier that can manufacture an electronics board or a software vendor that has developed a component with a particular algorithm.

The pattern for an acquired component proceeds with developing a concept for what is needed and a specification for the component. The specification is the basis for a request for proposal (RFP), which is sent out to potential suppliers that are expected to offer potential solutions. The suppliers in their turn use the specification to develop a design, which might simply be an off-the-shelf product or might involve development work on their part. Once the suppliers have a design, they respond to the team. The team evaluates whether the design in fact meets the specification and determines which option is best, if there they have more than one potential choice. In many cases the team will build a simple prototype using a supplier’s prototype implementation, if they have one, as part of the evaluation. After that, the supplier implements, builds, and delivers the component. In other words, this pattern moves the design and implementation work away from the project team and onto the supplier.

The team, however, still does some amount of verification once they have received the implementation. This acceptance testing may be more limited than it would be for a bespoke design, if the supplier provides information about the verification steps they have taken. Nonetheless, the team should spot check any verification work that the supplier has done and must check that the supplied component integrates as expected into the rest of the system.

Acquiring components like open-source designs or software do not have to go through the process of developing a formal RFP. However, these components do still require evaluation before deciding whether to use the design or not. The team must ensure that the license terms are compatible with the system. The team must also ensure that the potential component meets the specification of what is needed of it. Finally, the team must evaluate the quality of the component—which for open-source components, includes not just the quality of the artifact itself but also its governance and supply chain security [Goodin24][CVE24].

This pattern involves support roles that I have not detailed out elsewhere. For example, the acquisition might involve someone who manages contracting or payment. The acquisition will likely involve checking that the license terms and intellectual property rights associated with the component are appropriate for the system the team is building, which may require legal expertise.

Hardware components. Hardware development has different constraints than some other kinds of component development, and so a different development pattern applies. The primary cause of the differences is that a hardware component involves physically building one or more artifacts, which can take time and resources. This makes iterating on a design to work out bugs or to change features much more expensive than it is for software or higher-level designs. In addition, some verification testing is destructive, putting a component in increasingly harsh environments or under harder loads to determine when it fails.

Hardware development also differs from other kinds of component and feature development in the way terms like “design” are used. A design for an electronics board is a full description of how it is to be implemented; in some cases, it can be sent to an automated production system to create a complete physical board. Similarly, many mechanical designs are complete enough to send to a CNC machine or additive printer to create the physical artifact. By comparison, a software design is more abstract; it cannot be directly translated into a working program. Software source code is closer to mechanical or electronic designs, as source code can be sent to compilation tools that produce the executable artifact.

These constraints have led to disciplines about how to organize hardware development. I discussed the EVT/DVT/PVT pattern earlier (Section 23.4.1), which defines a sequence of phases for developing and verifying a hardware component. The NASA approach uses different language [NASA16, p. 124] to describe the sequence of hardware artifacts to be developed and verified. The two approaches are similar, with one naming the phases and one naming the artifacts.

This approach splits up the design, implementation, and verification phases into multiple iterations. There are typically four iterations.

1. Preliminary development: produces a breadboard or brassboard, which are low to medium fidelity versions of the component. This version is focused on function but often has a form unlike that of the final. It may use off-the-shelf components that will not be used in the final version. These may be subscale or digital models. This step may be repeated multiple times, adding features at each iteration.
2. Engineering unit development (or EDU, engineering demonstration unit): produces a version that closely resembles the final version in both function and form. It is put through EVT (engineering validation and testing), which verifies that the version mostly meets its specifications. The engineering unit may have a small number of defects, but at the end of verification the team should have confidence that these can all be corrected to produce the final version.
3. Qualification unit development: this produces one or more units that are built to the final design. These units are typically built manually in small numbers. They are put through DVT (design validation and testing), which involves thorough testing of the function and form of the component. For some systems, some of these units will be tested to failure in order to show that the design can function correctly across the full range of environments in which it will operate. Aircraft wings, for example, are tested by flexing them until they fail. Spacecraft electronics are subjected to heat, vacuum, radiation, and vibration beyond what they will experience in use, and those tests are likely to damage the components. These units are also used for certification with regulators or similar industry organizations.
4. Production or flight unit development: this step produces a small number of components that can be deployed into operational systems. These are used to verify final manufacturing processes, using the PVT (production validation and testing). This includes matters like supply chain operation, manufacturing logistics, and delivery and storage of the final units. These units are put through acceptance testing, which verifies that the manufacturing process builds components that are identical to those built for qualification.

Figure 27.2: Hardware development phases

The fourth step, producing production or flight units that can be deployed, can occur as part of development or later, in a production phase after the system has been accepted (Section 28.1). If a component is going to be mass produced, verifying the manufacturing methods is worth doing before declaring that the component is complete. After acceptance, the manufacturer will build more units. On the other hand, if only a handful of units will be built and they are expensive to build, such as with individual spacecraft, delaying the production of those units until after acceptance can manage risk.

Finally, the development of a hardware component is part of the development of the larger system. This leads to two ways that the hardware development steps can be organized, depending on how the hardware development will be synchronized and integrated with other parts of the system.

The first way is to plan out the hardware component development as its own thread of work. This way has the advantage of keeping the team focused on designing and building the component.

The second way is to break up the hardware development thread into smaller steps, and put some or all of those steps in feature development threads. For example, when building a circuit board that will run a control system, it will be hard to verify that the board works without some version of the software that runs on it or the interfaces to sensors and actuators of what it controls. In other words, verifying the integration of the hardware component with other parts of the system is an essential part of checking that the component actually works. This is the way virtually every project I have worked on has actually planned out its hardware development work.

As an example, this sequence of feature development steps is loosely based on two different control system implementations in projects I have worked on. The sequence shows how different hardware and software components come together to implement increasingly complex features. This approach integrates the hardware and software parts in incremental steps.

27.9 Acceptance

The acceptance phase is the time for final checks that the developed system is indeed ready for production and deployment. It is the last step in the overall system development life cycle.

There are three kinds of checks involved: that the system can be put into production and deployed; that the customer (or their surrogate) validates that the system is what they need; and that regulators approve the system, if needed.

The check for production and deployment involves verifying that the manufacturing and distribution process is ready for operation, and that all the procedures and tools are in place to install a manufactured product for customer use. For a software-only product, the manufacturing and distribution procedure involves packaging the software release and putting it on distribution servers (or manufacturing distribution media if it is not distributed over networks). The deployment readiness involves verifying that the packaged software has prominent and understandable instructions on how to install it and start using it. On the other hand, for a mass-produced hardware product, verifying manufacturing and distribution involves checking that the manufacturing line can correctly build the system, that it has the proper supply chains in place to support the manufacturing, and that the products can be shipped and warehoused before delivery to customers.

Validating that the system meets customer needs involves customers trying out an instance of the system—not just looking at documentation about the system. This often involves getting one or more customers to use a test installation of the system to do the tasks that the customers need. For some systems, this kind of validation can be done by beta testers, who are given an almost-ready version of the system and try it out in their environment while providing feedback of what works or doesn’t. Other systems that involve more installation and setup can involve setting up test installations that the customers come to use.

Regulatory approval involves different procedures in different industries. An aircraft, for example, must be reviewed and certified by the appropriate civil aviation authority. A spacecraft mission typically requires licenses for launch, communication, and certain kinds of earth observation. Other systems may need approval by an industry safety organization. Most of the work to get these approvals or licenses is part of the development phase, and the acceptance phase is the final check that the necessary approvals are in place.

Once these checks are completed, the final milestone is for the organization and the project to decide whether to proceed to production and deployment or not. Many systems are designed and built, but in the end the organization behind the project decides that the result does not justify the investment in production. Many commercial aircraft, for example, are designed and built, but in the end there is not sufficient sales interest to start production and the aircraft model is quietly retired.

Chapter 28: Operation

Once the system has been developed and verified, it is ready to be manufactured, deployed, and put into use. The initial work of building is done, but there is much more to go. There are several ways the operation phase can proceed, depending on the kind of system, kind of customer, and the role that the organization that developed the system plays.

The general flow is first to manufacture or produce the system using the artifacts that have been developed, then deploy instances of the system. After that, the system instance is in operation. Further development of the system, to evolve it or to fix problems, continues in parallel with customer operation. Finally, at some point, the customer will decide to retire and dispose of the system instance. The steps of deploying, operating, and retiring system instances can occur multiple times in parallel for different customers.

28.1 System production

The production phase covers manufacturing the artifacts to be deployed.

Bear in mind that this is a brief overview of manufacturing, intended to explain the main points that people like systems engineers or project managers will need to know in order to understand the general scope of the work, and to understand how the manufacturing steps are related to other parts of the system-building work. Manufacturing has been studied and refined for a couple centuries, and there is an extensive literature with far more information.

There are several kinds of production that different projects might use. These include:

• One or small batch production, often done mostly in house. For example, a project might manufacture one spacecraft and a backup for a mission, or a company might build a one-off system for internal use.
• Mass production, often by outside manufacturing lines. Physical consumer or office products use mass production.
• Digital systems with little or no physical artifacts. For example, software that people will download over networks.
• Hybrid, with physical artifacts and digital added to them. This is the most common today: systems that have both physical artifacts that are manufactured and software or configuration that is loaded into them, and that software may be updated in the future.

Production of a new system for a new installation can also differ from production of parts for maintaining or upgrading an existing installation. A new system might consist of a complete collection of hardware components that will be assembled from scratch for the installation. Producing replacement or upgraded parts, on the other hand, consists only of manufacturing a few parts and making them available for deployment into existing installations.

A review and approval to begin production milestone checks that the project has everything ready before committing to production, as discussed below. The review checks that the system development has completed all its milestones and that a system will be ready to deploy when manufactured. It also checks that everything needed for production itself is ready: the manufacturing tools and people, suppliers, testing. It also checks that the organization is prepared by being able to pay for supply and manufacture, and that people are ready to deploy systems once their parts have been manufactured, so that capital does not remain tied up in unneeded inventory.

Production relies critically on security of the supply chain, management of the developed artifacts, the manufacturing process, and the delivery mechanisms. All these elements of the production process have been attacked in recent years. For example, the SolarWinds attack [Zetter23] compromised the production process for their software, which was then distributed to and installed by many other organizations and led to attacks on those other systems. There are other reports of fake hardware components (e.g. pressure sensors [Control19]) being injected into a supply chain. These attacks can result in loss of system components, delaying deployment to a customer, exposure of intellectual property, deployment of a faulty or dangerous system, or creation of security problems for the system’s customer.

The overall production process has the following steps:

• Take in implementation artifacts, input stock supply, and manufacturing procedures.
• Acquire input stock from suppliers or inventory.
• Perform manufacturing procedures.
• Perform acceptance tests.
• Record tracking information for each manufactured unit (such as serial number, build history, or test history).
• Deliver deployment-ready artifacts to inventory or directly to customer.

This flow depends on the supply chain of parts used in manufacturing or production. Any physical parts or stock used must be on hand to perform manufacturing; this implies that the stock is in inventory, and that it has been supplied from some qualified source. Sourcing implies selecting the suppliers and setting up contracts for them to provide the stock. The contracts with the suppliers should include clear specifications of exactly what stock or components are to be supplied, along with evidence that the delivered parts meet the specification.

Procedures for receiving materials from suppliers and maintaining inventory are part of the definition of manufacturing procedures. The procedure will typically need some amount of space for maintaining this input stock, along with managing information about what stock is on hand and what should be used next. The storage space maintains the input components or stock in an environment that will keep the material in its designed storage conditions. The procedures include determining when to order more stock. The receiving and storage facilities should have security that ensures that material is not stolen or replaced.

The production process relies on accurate configuration or version management. The artifacts used to manufacture the production components should have consistent versions, and those should match the versions used for final verification during development. If inconsistent implementations were manufactured, the components might not work together—and the resulting problems are often subtle.

The manufacturing procedures specify who does what steps, in what order, using what tools. These procedures are designed during system development and verified during production verification testing (see the section on hardware development above).

After system components have been manufactured, they are checked to ensure that there are no manufacturing defects. This is typically called acceptance testing. For many hardware components, this involves putting the component through a set of tests that are defined during system development. These tests do not stress the component to a level that will induce faults, like testing at high temperatures or voltages; the tests only look for potential manufacturing problems. Some mechanical or electrical components go through a “burn in” period, which operates the component long enough to catch early component (“infant mortality”) failures. For some other kinds of components, only a sample of each batch of components gets tested, under the assumption that manufacturing defects will tend to cluster in one production batch (for example, one day’s production shift).

The production process involves a significant amount of record keeping. Each produced component has its own set of records. These records start with the component’s identity, typically represented as a serial number. The record identifies what version of the input development artifacts were used, often by associating a release version number or code with the serial number. The records include when, by whom, and using what equipment the component was built, so that if parts start failing an analysis can identify other components that may be at higher than expected risk of failure. The records track what parts or stock were used to manufacture the component: the serial number of components used, if appropriate, or the supplier, model, and batch number of stock.

In addition, each manufactured component must be identifiable. That typically means that it should be clearly labeled with its model or version information and serial numbers, at minimum. The labeling is typically in both human- and machine-readable forms.

Once a component has been manufactured and checked, it is placed in inventory and later delivered for deployment. The components in inventory are stored in secure spaces that maintain the components in their designed storage environment—often dust-free, within a particular temperature and humidity range, and so on. The inventory is managed to know what components are in stock and ready to send for deployment.

The production process needs to be resilient to disruptions. One company I worked for was building hardware systems outside the US, and investors asked the company how they would handle a political or military disruption in that country. (The answer was that the company would go out of business because it had no alternative manufacturing option.) Many production or manufacturing processes are also in places that can be vulnerable to natural disasters, including earthquakes and storms.

Finally, the manufacturing process is generally a human process, and processes involving humans have a tendency to drift over time away from their originally-intended procedures (see e.g. Leveson [Leveson11, Chapter 12]). This drift can come from changes in how people are trained, people finding potential simplifications in the procedures, changes in the environment in which the people are working, and many other causes. The designs of robust, safe manufacturing procedures include periodic audits to check that people are performing the procedures as originally designed, and to re-design the procedures if they are found to have problems in use.

28.2 System production examples

I mentioned earlier that different projects follow different kinds of production patterns. Here are a few examples that show some of these different approaches.

28.3 System deployment

The objective of deployment is to set up a system instance for a customer and get them successfully using that system.

There are several kinds of deployments. The first variation is: who is doing the deployment? Consumer products are set up and installed by the customer. More complex systems are delivered and set up by a team that is part of the project. I will refer to this as “assisted deployment”. Other systems are deployed and used internally by the organization that created them. The second variation is whether one is deploying a complete new system installation, or installing an upgrade into an existing system.

The overall flow of events is the same for all these variants:

1. Deliver and install the system (or its new components) at a customer’s location;
2. Verify that the installed system works as expected;
3. Train users on how to use the system properly;
4. Migrate any data or material if needed; and
5. Put the system into operation.

A system is deployed into an environment. That environment might be a customer site for a physical system; it might be spread over multiple sites; it might be an attachment to a launch vehicle; it might be resources on a compute server somewhere. In all those cases, the customer finds the places where the system can be installed. The deployment team and the customer usually interact before deployment starts to let the customer know what is required for the system, and for the customer to let the deployment team know what is available.

The environment for a software system might include the number and kind of compute servers used, the amount of memory or storage on each, the reliability and security of the servers, and the reliability of each server.

The environment for physical systems might include physical space, along with the temperature and atmosphere in that space. It might include the mechanical mounting needed, along with electrical, water, networking, and other supply lines.

Some customers will be migrating from an existing system to the new system being deployed. The migration might include moving information from the old system to the new system, or it might involve moving physical artifacts or supply from one to the other. Developing the migration procedures are a development activity on their own; in effect, they are a second mini-system to design and implement.

Complex systems will have users who need to be trained in order to operate the system safely and correctly. The initial group of users are trained during deployment, so that they can verify that the system works correctly and can take over its use once the installation has been accepted. Other users will learn to work with the system later, perhaps years later.

The installed system includes education and training materials for these users. These materials are assembled during the development phase of the project.

Different kinds of users may interact with the system. At the simplest, there are users who directly command and use the system’s primary behavior. A system may also have administrators who are responsible for specialized tasks, such as managing the set of users or the system’s security. It may have people who are responsible for maintenance and repair. I likely has other people who set policy for how the system should be used. All these people use the education and training material, and that material must address each of their needs.

Deployment presents a number of ways that someone could attack and compromise the system. The system components will be in transit from the production facility or warehouse and could be tampered with; they will be received at the customer site and might be accessed before being installed. The system components may be partially installed but not fully configured to be secure during the deployment process. The deployment procedures themselves could be altered or hijacked. All these potential exposures mean that the deployment procedure must be designed with security in mind, and that security must be evaluated as part of the system requirements.

Deployment includes setting up the customer on a customer service system. Once the system has been installed and the initial users have been trained and given access, the customer will begin to take over system operations. As they do this, they are likely to find they do not actually understand some parts of the system and have questions. They will use the customer service system to communicate with the project team for questions and to report problems.

Accidents and incidents may happen during system operation. When these happen, the customer works with the team that developed and maintains the system to investigate what happened. If the accident is serious enough, regulatory agencies may be involved. During the deployment process, the team establishes the necessary working relationships with the customer that will help the customer to detect when accidents have happened and to bring in the team for investigation. The investigation may determine that there is a flaw in the system, in which case a problem report and change requests are sent to the team to guide fixing the flaws. Section 28.7 below addresses how the team handles such changes.

Setting up the customer for ongoing success using the system is the last part of deployment. Once the system is in operation, the customer’s users are responsible for the ongoing safe and secure use of the system. Users of complex systems tend, over time, to find shortcuts and workarounds for how they use the system. They may forget part of their training, and new users may not be trained fully correctly. The environment in which the system operates may also change—parts might be moved, air conditioners changed out, or electrical feeds changed, for example. All of these can slowly change how the system is working and lead to accidents. Regular monitoring or auditing of system and user behavior is necessary to detect and correct these drifts and avoid accidents, and this auditing must be backed by management policy and actions. (See Leveson [Leveson11, Chapter 12] for background.) The deployment activities, therefore, must include working with the customer to establish the necessary monitoring activities and to establish necessary management policies.

Customer deployment. The components for these systems are delivered to the customer, who is responsible for installing or upgrading the system. The process includes:

• Delivering the system, along with deployment procedure instructions and training material.
• The customer sets up the environment for the system. This might involve creating space for machines, providing power, providing network connections, providing cooling, and so on.
• The customer follows the deployment procedures to install and test the system installation.
• The customer might contact the project’s organization for assistance, but most customers should be able to follow the instructions on their own.
• The customer migrates any materials or information from an old system. The customer is generally responsible for determining how to do this, though the system deployment instructions might include procedures for common cases.
• The customer puts the system into service.

Assisted deployment (internal or external). When someone from the project team does the deployment, the process is similar to customer deployment.

The process includes:

• The team gives the customer requirements for the environment the system will require, such as server room, logistics bay, assembly bay, or an open field. The customer is responsible for arranging that environment before deployment starts. The team may verify that the environment is ready before starting delivery or deployment.
• The team delivers system components to the customer site, then assembles them following deployment procedures.
• The team performs basic testing to ensure assembled system works.
• The team provides training on operation and maintenance to the customer.
• The team and customer may work together to migrate information or materials from a previous system.
• The customer performs acceptance testing, and accepts the deployed system when no issues are found.

28.4 System deployment examples

Deployment follows many different patterns, depending on the kind of system and customer. The following four examples illustrate some of the range of ways that the general deployment step can happen.

Complex system, shared deployment responsibility. The previous examples have been simple, performed entirely by the customer. The next example covers a more complex deployment.

Consider an information system that supports a repair and maintenance workshop. This example is based loosely on a system I worked on for local government public works agencies, which maintained a wide range of equipment from buses to lawn mowers to backhoes.

The repair and maintenance organization had multiple shop sites. Some shops were specialized for working on particular kinds of equipment.

The system provided record-keeping support for managing work orders (repair orders), scheduling resources like work bays or large equipment, and managing parts inventory. It also interfaced with the customer’s other IT systems: security and user authentication systems, and systems to place orders to buy parts and to pay for them.

For a particular installation, the customer asked for a set of features to be added to an existing software package. The development phase of the project for this customer involved work to determine their specific objectives and changes, implement changes to the base system, and then validate the customized system with the customer. Once the customer accepted the changes at the end of the development phase, a production phase generated the software installation packages and other materials for the deployment.

Physically, the system consisted of a small set of servers in a server room, plus workstations of different kinds at the workshops. This equipment used communication equipment between the sites and the server room. The server room provided power, cooling, communications, and support services like backup and security for the servers.

The customer wanted to perform a phased installation and roll-out, where initially only a few people would use the system and over time its use would be extended to more and more sites. The goal was to minimize risk by avoiding disruption to the shop’s existing work, and to contain any problems that might come up as the shop users learned to work with the system. A phased installation would also allow the customer and deployment team to monitor the performance of the servers and communication systems in order to identify unexpected behaviors before they caused problems. The customer decided to continue using their existing (paper-based) system for all existing work, so no data would be migrated into the new system.

The project’s deployment team was responsible for installing and configuring the initial system, and for training the initial users. The customer installed servers and communications, along with workstations at the shop sites. The customer was also responsible for adding users to the system and would take over training and configuration after the system had been rolled out to half the shop sites.

The deployment process proceeded as follows:

• The customer installed servers and networking to host the shop management software.
• The deployment team installed the software and verified that it was running correctly.
• The customer installed workstations for the system administrators, and the deployment team trained the administrators.
• The customer installed workstations at one test shop site. The deployment team trained users at that site. The customer’s IT staff and the deployment team helped the new users and monitored system performance for the one shop site.
• The installation and training process continued with a second site.
• After half the sites were set up and users trained, the customer accepted the deployment and took over responsibility for operations. The customer installed workstations at more shop sites and trained the sites’ users.
• The deployment team remained available to help with questions and problems under a support contract.

This system thus had a phased transition between deployment and operation, rather than a hard split between one phase and another.

28.5 System operation

In this phase, the system is placed into operation. The customer uses the system, performing administration and maintenance as needed. Most of the system operation is the customer’s responsibility; in this section, I focus only on what the project does to support the customer’s operation.

The system operation phase affects the project team in two ways. First, the team will sometimes support the customer during operation. Second, point of the team’s work is to build a system that can go into operation, which means that the system’s design supports all the activities that the users will do. This includes the rare and exceptional activities, not just everyday usage, so these activities are included in the concept and specification to which the system is built.

The customer is responsible for maintaining the system. That may mean only following procedures for periodic checking, but for many systems maintenance can be far more intrusive, and involve regular replacement of some components. The customers rely on maintenance procedures that are designed as part of the system to keep the system operating safely; these maintenance procedures are designed to take safety, security, and reliability constraints into account. The customer also periodically orders replacement parts to install into their system.

The customer also takes care of their users. This includes adding and removing user access to the system and training those users. The project team supports these tasks by including features to manage users and their roles as part of the system. The team also develops training material that the customer uses when bringing on new users.

The system may have problems from time to time. These may reflect flaws in the system, improper usage, wear and tear, or combinations of all three. The customer, as the system owner, is responsible for handling the problems. However, the project team sets the customer up to be able to address problems by developing instructions for detecting and diagnosing problems, and training some of the customer’s staff on how these work. The project may also provide services to help diagnose and repair problems. The project also provides some form of customer support that the customer can use to report problems back to the project.

Most complex systems have human elements—users who operate the system and in doing so act as a control system that manages system behavior. As I noted in the previous section, these users can change how they interact with the system over time, finding shortcuts or using the system in ways they are not expected to. The customer establishes usage policies and performs monitoring and auditing tasks that check that people continue to interact with the system in safe and secure ways. The project team sets the customer up to perform this work by documenting what constitutes safe and secure system usage, including the rationale for why some interactions are acceptable and others are not.

Accidents happen. When some loss or injury occurs due to the use of the system, both the customer and the project team have a responsibility and interest to determine why the accident occurred in order to avoid future accidents. The accident investigation may also be mandated by regulation, in which case regulators are involved. The customer may be able to pursue the investigation on their own, if they have sufficient information about how the system is supposed to be used safely. The project assists in the customer’s investigation by providing that information, which includes the documentation of how to use the system safely, and why. However, for serious accidents, the investigation often requires a more in-depth understanding of the system’s design and implementation. The project prepares for supporting these investigations by maintaining complete records about the system’s concept, specification, design, and implementation, including explanations of the rationale for why choices were made and safety or security analyses that the team did about the system’s design.

Finally, the customer may find that their needs change over time, or that there is some aspect of the system that does not work as well as they had planned. These changes can be externally driven; for example, regulatory changes that affect the customer’s industry can affect what the customer needs from the system. The project team can receive change requests (along with problem reports) through a customer service mechanism.

28.6 System operation examples

Operations vary widely depending on the kind of system. Here are some examples illustrating the range.

28.7 System evolution

The system evolution phase is about making changes to the system after it has been released and potentially deployed to customers. These changes can happen for many different reasons—such as a planned roadmap for adding to the system over time, requests for changes from customers, fixing problems, or changes in regulation. System evolution can occur in parallel with system deployment and operation.

Overall, system evolution is a recapitulation of system development (Chapter 27). It involves working out a purpose for the change, a concept for how the system will work when changed, leading to specification, implementation, and verification. These steps use information about what has already been specified and implemented in the system, along with the reasons why it is that way, to work out how to make changes that achieve the desired results without disturbing the system’s existing behaviors.

Making changes starts with a change request. In whatever form the request takes, it identifies who is asking for a change, what their purpose is in the change, and why it is worth doing. In practice change requests are usually maintained in a database. Requests can come from many sources. They may be part of the project’s long-term plan to continue developing the system. They may come from customers, who ask for new or changed capabilities. They may stem from the investigations into reported problems or accidents, in order to avoid problems in the future.

A project does not act on all change requests. Some of them will be technically impossible; some will be infeasible because of time or resources. Others might be reasonable requests that have to wait until higher-priority requests have been addressed. The team looks at each request received to determine its importance, its feasibility, and its cost, and makes decisions about whether to accept or reject the request based on the analysis. If a request is accepted, the team determines a relative priority compared to other work or a potential deadline. These are used in planning the team’s upcoming work (Section 20.5).

Determining whether a request is feasible involves determining how much of the system will be affected by a potential change. While working out the concept for the change, a team member determines what parts of the system will be affected, using documentation about the system’s structure and design (Chapter 12). The result is a preliminary analysis listing the set of components that will be changed and the general nature of those changes. This information is then used to estimate the effort that will be needed to design and implement changes for the request.

Changes happen iteratively. There may be multiple iterations in progress concurrently if multiple changes have been accepted. Handling multiple concurrent iterations requires careful configuration management discipline (Section 17.4).

Making the changes involves changing the specifications and designs for affected components. These changes can be difficult to make accurately because they are done to an existing, complex set of relationships between components. Making a change without causing flaws depends, then, on being able to accurately understand the structure of the system and how parts of that structure contribute to emergent properties like safety constraints. This relies on having rationales, analyses, and earlier designs available, so that people can work from an accurate information base.

Once a change has been specified, designed, and implemented, it is verified. Verifying the work for a change has two parts: ensuring that the modified system meets the new specifications and the purpose of the change request, and ensuring that the rest of the system continues to work correctly.

Once the changes have been verified, they can be deployed to customers as an upgrade or incorporated in new deployments, using the production (Section 28.1) and deployment (Section 28.3) patterns already discussed.

The team continues to evolve the system until the team is relieved of responsibility for fixing problems or when the system is taken out of operation.

The overall process for system evolution includes:

• Receiving change requests. These may include problem analyses, such as root cause analysis or accident investigation reports.
• Triage the requests to determine the importance of addressing each one. Perform a basic analysis of the affected scope and the feasibility of a change.
• Decide on which change requests to address and which to reject, and the relative priorities for requests to be addressed.
• Reiterate the development pattern.
  • Update the purpose and concept to include the changes. Identify what parts of the system will be changed to address this as part of the concept.
  • Modify specifications and designs.
  • Update implementations.
  • Verify the changes. This includes verifying manufacturing changes, training changes, maintenance procedure changes, and similar material.
  • End with acceptance test. This might include a beta test with customers.
• When development is done, reiterate production and deployment. Manufacture the update packages: new hardware components to be deployed, software updates, new procedures and training. Deploy the updates, either by moving in new hardware or applying digital updates, along with training.

28.8 System evolution examples

28.9 System retirement

No system lives forever, and most are deliberately taken out of service when their usefulness has ended.

Most systems continue in operation until there is a decision to retire them. For some systems, this comes when the purpose for the system has been completed—for a spacecraft mission, for example. For others, it comes when the system has worn out enough that ongoing maintenance and repair costs outweigh the cost of replacement, such as for vehicles that wear out. Yet others are replaced because newer systems become available that can meet the customer’s need better.

A system being retired and disposed of typically goes through three periods. In the first period, the system is in normal operation, but the decision has been made to retire it. During this period people plan how to shut the system down and transition its functions or information. They should conduct dress rehearsals to verify that the procedures will work as expected. The system then enters the second period, where it is no longer in normal use but may remain at least partly operational to support transition and archival. Once those are verified complete, the system is shut down for the last time, is dismantled, and its resources are disposed of.

There are two primary aspects of retiring a system to consider: what to do with information or materials that should be migrated to a new system, or archived, and how to dispose of the artifacts that make up the system.

I discussed migrating into a system in Section 28.3 above. The task of migrating out of a system is part of the same process, involving developing a plan for migrating information or materials from the old system and into the new.

There will be other information that people will want long after the system has been retired, in many cases. This can include logs of system activity or user access that may be needed for later accident investigations or legal inquiries. It can also include information or materials that the system processed that are not being migrated to another system, but that may be valuable in the future. This information is moved from the working system to some kind of archive. Developing the procedures to archive the information, how the information will be organized, and the system to hold the archive requires development on its own, just as migrating information from one system to another does. This development phase involves determining what information needs to be archived and how it will be used once it has been stored, which in turn leads to a concept, then specifications, then a design.

Archived information is usually retained for a long term. If a system has been used for business or manufacturing, retention is mostly governed by regulation—anywhere from one to 30 years in the US, depending on the kind of information. Scientific and medical data is often of value indefinitely, though legal retention requirements may be shorter. Scientific data is often re-evaluated decades after it was first gathered; for example, data collected from the Viking landers on Mars in the mid-1970s was re-interpreted thirty years later after other missions gathered more information about Martian soil composition [NavarroGonzalez10]. This particular example also illustrates a problem with many data archives: the mission data were recorded on microfilm and had to be scanned to get digital data to process.

Long-term archival media often have two problems. First, the media wear out and decay over time, which has led to information believed to be safely archived to be found to be unreadable [Purdy24]. Second, even if the media are readable, there may no longer be machines that can read them. I have a number of backup tapes for which I have not been able to find a drive to try reading them.

Sometimes physical artifacts are retained from a retired system. It is common to keep parts of aircraft and spacecraft in museums after they are retired, for example.

Disposing of system artifacts can range from trivial to complex. Erasing a software application and its data, for example, is easy; once the storage media have been erased, there is no further meaningful trace of the system remaining. Disposing of a system that processed hazardous biological or chemical materials, on the other hand, can be difficult.

The retirement and disposal procedures must be secure. An unauthorized attempt to shut down a working system can cause major losses, and can lead to safety hazards. Information and materials are being moved around during migration and archival, and are potentially accessible to being copied or corrupted. Physical artifacts that are being decommissioned can carry confidential information about both the way the system works and about the customer that has been using the system.

28.10 System retirement examples

There are many different ways systems are retired. Here are three examples that illustrate different approaches.

Chapter 29: Project ending

When a project ends, there are three objectives: completing obligations and support for stakeholders (Section 16.2); saving information and artifacts that might be needed in the future; and releasing resources that the project used.

Note that ending the project is separate from retiring any particular instance of a system. Ending the project is about stopping development and support for a system product, independent of whether there are instances of that system in operation or not. Some projects will combine these, such as for exploration space missions that build and fly one spacecraft.

A project might end for one of many reasons. It might have a fixed term or have completed a defined system deliverable. It might run out of money or time. It might no longer fit the organization’s or funder’s strategy, perhaps because a better replacement system is planned. Competitors might have won over customers and there is no longer demand for the system. The team might be unable to deliver, with the project behind schedule or over budget or lacking key features.

The first step is a decision to wind down the project. This is typically a decision made by the organization that hosts the project, or its funders; the project staff generally do not make the decision on their own.

A decision to end the project is followed by a plan for how to do so, which defines the steps the team will take to meet the final objectives. The plan typically gets review and approval before proceeding. (In some environments, at least part of the plan must be worked out early in the project, long before any decisions are made.)

The following sections list some of the steps that are involved in ending a project. The specific steps will depend on the project; for example, not all projects have contracts with funders that must be closed out. The team can use this list to help build the plan, bearing in mind that some steps should be done in particular orders. Customers should be notified of the project’s impending end before ending contracts; shutting down production should happen only after all upgraded components have been manufactured.

29.1 Project cancellation

Some projects end because they are canceled, even before they have completed their development phase. Anecdotally, it seems that more projects are canceled than go to completion—this is a consequence of using competitive approaches to programs, and the net effects of competition are generally regarded as valuable. The information in this chapter applies to canceled projects just as to other projects.

Consider two examples, based on projects I have worked on.

In the first project, the team was writing a proposal for a US DoD spacecraft system. In the proposal-writing phase, the team has to establish the basic architectural and management approaches for the project, show they meet the department’s needs, and establish the price at which the team proposes to build the system. The team progressed through establishing the initial concept and architecture for the system, and we began evaluating the solution to see how good a job it would do for the customer and how much it would cost to build it.

We had a checkpoint milestone where we reviewed what we had found. At that review, it became clear that while our team had a decent solution for the needs, we did not have a great solution, and that other companies we expected to propose designs would likely have better solutions (because they had more experience in a couple of key technical areas). We made the decision not to pursue the proposal.

This was a good decision. Assembling a proposal is not a small task; we had a team of about 15 people working long hours. For US government projects, the proposer generally pays for the proposal development. Choosing to spend our team’s time and money on this project meant that the team couldn’t work on some other project. We judged that the opportunity cost was not matched by the probability of successfully winning the contract, so we freed up the team to work on a different system that did prove successful. If we had continued to work on the original proposal, we would have spent the budget available to develop proposals and could not have spent it on the proposal that succeeded.

In the second example, a different US DoD spacecraft program, the team was about two years into a multi-year contract. The team had performed excellently in a competitive first prototyping phase, and was the only team to be selected to move on to a second phase for building an initial working version. A key subcontractor on the team had staffing and management problems, and were not delivering results. Within the team we were struggling to fix the execution problems or find another way to build the necessary components, all the time keeping a large staff on payroll and running through budget. While the technological solutions for many system capabilities were probably sound, the team could not deliver. The customer observed the problem, and after working with the team to try to resolve the problems, went through the process to cancel the project.

This was also a good decision. In hindsight, the team lacked necessary capability in the subcontractor and in the project management team. If the project had been allowed to continue, it is unlikely that the team would have solved the problem and more money would have been spent without benefit in the end.

The take away from these examples is that there are many sound reasons for canceling a project. Sometimes the cancellation is designed in (as with competitive acquisition); other times it is because continuing to invest money, time, and the care of the team building the system has become unlikely to pay off.

For a more general discussion of US DoD project failures, see the report by Bogan et al. [Bogan17].

Chapter 30: Using the reference life cycle

The last several chapters have presented a reference life cycle pattern. This pattern is intended to inspire thoughts about how a project can organize its own work. It is not in itself ready to use off the shelf. Each project will have its own needs, its people will have preferred ways to work, and some projects will have life cycles mandated by regulation or industry standards to follow.

XXX purpose of the life cycle – get a system built, deployed, and sustained that meets customer and other stakeholder needs – in doing so, develops the whole system including all its development artifacts, not just the end product – plea for thinking about flexibility with discipline, and the idea of the artifact edifice

The reference pattern does not discuss the roles involved. A full definition of each phase will include definitions of who performs different tasks in each phase, and in particular who is responsible for milestones. I argue elsewhere (e.g. Section 8.2.6) that, to be meaningful, reviews must be done by people with an independent perspective on the material being reviewed, and that approvals must reflect a check on the work fitting into the project’s big picture.

The objective for any project is to develop and adopt a life cycle that meets its needs. In the next section I discuss several principles that a good life cycle will follow; these can help people evaluate a life cycle they are considering. Some other considerations include:

• Is this project intended to deliver a system once, perhaps with upgrades and fixes after delivery, or will it evolve the system through multiple releases for a long time? A system that is expected to have multiple releases needs more thinking about how to sustain the development over time and how to support customers at the same time as developing new releases.
• What make-or-break events are foreseen during the project? These include things like getting funding, aircraft type certification, or spacecraft launch. The life cycle should include these milestones explicitly, and include the work to be done to prepare for them. In many cases these will interrupt what might otherwise be a continuous development flow. (See Section 30.2 below for examples.)
• What regulatory constraints are there on the project or the system? Are there work products or milestones needed to satisfy regulators?
• What development methodology will the project use?
• Is the system intended for a single customer, or is it meant to be a product marketed to multiple customers? If there is a single customer, they can be included directly in the life cycle—especially if the customer is internal. If there are to be multiple customers, a marketing team may have to stand in for customers, and they may need time to investigate particular market segments before they can accurately reflect potential customers.

The project works out what life cycle patterns it uses, and documents the patterns. This effort starts during project preparation (Chapter 25). It does not necessarily need to define the entire life cycle all at once; it can be done iteratively, as long as the work keeps ahead of what the team needs. In practice I have found that enough should be completed in the project preparation phase that the team understands the general complexity of the work ahead, has chosen a development methodology, and can name major milestones they will need to meet. The remainder of the life cycle can be worked out during purpose and concept development and likely will be refined or adjusted as the project moves along. (I worked on one project that had a limited budget, and spent much of that budget writing elaborate management and engineering plan documents before even beginning to work out the high-level system concept. The result was a pile of such documents that were never looked at again, which was a waste of their efforts.)

In no case should the team get ahead of the defined life cycle. See Section 8.1.5—Principle: Team habits for a discussion of this principle.

The life cycle patterns have value only if the team actually uses them. This means that the team must know that the patterns exist, understand them, and agree that they are useful. The people in the team must also understand that they have a responsibility to follow the patterns, or to raise an issue when they find a problem with the life cycle’s definition. Achieving these means educating people as they join the team about what the life cycle is and how to learn about it, as well as monitoring that everyone actually follows the patterns. The team can also learn about and accept a life cycle a bit more easily if they are involved in developing the patterns; at minimum, they should be able to give feedback before the patterns are adopted.

The life cycle patterns are documented in a way that the team can find them and learn about them when they are joining the team and when they need to refresh their understanding of how some step works. The documentation is an artifact that should be managed using the principles in Section 17.4: it should be versioned and under change management; it should be stored in a way that the team can find it when needed; and it should be secure enough that it will not be tampered with.

There is no one right way to document them, as long as the documentation is well-organized and accessible. Some organizations prefer to define the life cycle in a prose plan document, which can be printed in its entirety if needed. I have had some success maintaining the documentation in a wiki or in a collection of web documents; the advantage of these is that they allow linking between parts of the document. The patterns should be explained and listed explicitly; they should not be hidden in a workflow system that doesn’t let team members see and understand the whole context for their work (see Section 4.6 for an example).

The documentation for each phase or step in the life cycle should include the information listed in Sections 21.5, 21.6, and 21.7.

30.1 Meeting life cycle principles

In Section 21.10, I listed principles that a life cycle pattern should meet. The reference life cycle pattern in this part reflects these principles, though it cannot address all of them. Here are ways that a life cycle built using this reference as a base can address them.

30.2 Relation to NASA life cycle

As many people are familiar with the NASA life cycle, or may be obliged to use it (or a variant of it), in this section I discuss how the canonical NASA life cycle compares to this reference life cycle. I will use the general NASA life cycle defined in NPR 7120.5 [NPR7120, Figure 2-5]. I presented an overview of this life cycle in Section 23.2.1.

The NASA life cycle is divided into seven major phases:

1. Pre-Phase A: concept studies that define a potential mission.
2. Phase A: concept development that results in the definition of a feasible and useful mission.
3. Phase B: high-level design of the system for a mission and evaluation of the technology available for the mission.
4. Phase C: the bulk of development, in which the system components are developed, verified, and manufactured.
5. Phase D: assembly, integration, and test of the flight and ground systems; launch and initial on-orbit checkout.
6. Phase E: operation in flight.
7. Phase F: close out, including flight system disposal.

This life cycle was developed over several decades as NASA learned how to develop and operate complex missions. Elements of this approach have been adopted by many other organizations—terms like “System Requirements Review” and “Preliminary Design Review” have become nearly ubiquitous in the aerospace industry.

The overall flow of the NASA life cycle is organized around two constraints: fitting in with the US Federal funding cycle, and managing risk for a few highly expensive steps. The funding constraints come at the transition from Pre-phase A to Phase A, when the mission is approved and funded enough to develop its concept, and between Phases B and C when the agency commits to funding the full mission [NASA16, Section 3.5, p. 25]. The distinction between Phases C and D comes with Phase C covering development of designs and fabricating components, but actual assembly of a spacecraft does not start until Phase D, at which point there should be little residual risk that the system design will not work out.

The NASA approach was, however, developed for hardware-heavy systems and people who today develop spacecraft or aircraft that have a greater amount of software components sometimes find it difficult to map software project best practices onto the NASA approach. There are usually two issues: software development best practice puts integration earlier than the way many people interpret the NASA model; and many software developers combine design and implementation, especially for novel software functions. I show one way to reconcile these approaches in the mapping in this section.

The reference life cycle I have presented is organized around types of work—conception, specification, design, and so on. The NASA life cycle is organized at the highest level around milestones that check progress early, allowing corrections before committing agency resources. This means that the NASA life cycle splits several of the early phases in the reference life cycle in two, with a major review or checkpoint of the project’s progress before continuing. These two approaches are compatible: almost every project will have some kind of project-wide milestones alongside the milestones specific to the work phases.

In the following, I present how each of the NASA phases maps to the reference life cycle.

30.2.1 Outside the NASA life cycle

The reference life cycle defines the project preparation phase and project support “phase”. The preparation phase involves a rough definition of the project and establishing basic operations abilities. Project support covers support functions, like managing teams, finances, or artifacts.

In the NASA environment, the initial support is provided by one or more agency centers and external collaborators, using budget, tools, space, and people for general concept exploration. Each center has its own procedures for starting up a concept exploration project.

Similarly, the NASA agency provides essential support services to its projects.

In one project I worked on, the NASA Ames Research Center had a Mission Design Center that was charged with exploring potential mission concepts. A small group developed the mission idea and explored ways it could be realized. Ames and the agency provided all the key support infrastructure: staffing, finance, office and lab space, and IT services, for example.

30.2.2 Pre-phase A—Concept studies

The Pre-phase A work develops a concept for a mission, presumably in response to NASA agency priorities. It is expected to limit its work to the concept of a mission: what it might achieve, who would benefit from the mission, and high-level technical approaches that might support such a mission.

There is one major review in this phase: the Mission Concept Review (MCR). This checks that the potential mission is well formulated and that there is sufficient interest to justify funding “project formulation”—working out a detailed concept and high-level design.

At the end of Pre-phase A, after the MCR, the agency makes a decision whether to continue the project and fund it for “formulation”: the phases where the concept and high-level designs are worked out. This involves greater financial commitment than the early studies, and is the start of the “real” mission.

Pre-phase A maps to the purpose development phase (Section 27.3) and part of the concept development phase (Section 27.4). The purpose development phase covers identifying what the mission might do, and who the mission stakeholders might be. The concept development produces an initial sketch of a mission concept, without breaking the concept down into great detail.

30.2.3 Phase A—Concept and technical development

This phase is the first of two that are about developing a feasible high level design for a mission and ensuring that necessary technologies are available. Phase A includes developing a complete mission concept and high-level system designs. The team identifies any new technology that the mission will require and works out what will be needed for it to be ready to use in flight.

The depth of design and requirements is not clearly specified in the NASA procedural documents. However, my experience is that it is generally taken to include the spacecraft and its major subsystems, ground systems and their major subsystems, potential launch vehicles, and testing and other ground support equipment to a similar level. The exercise is intended in part to develop the general structure of the system and its likely cost, and in part to find those parts of the system that will require new technology.

Phase A includes developing a list of new technology that will be used for the mission, an evaluation of its maturity, and plans to develop that technology so that it will be mature enough for flight.

This is the first phase where a NASA project is funded for itself, as opposed to using resources allocated for general mission concept development. The various management and development plans required by NASA procedures get developed in this phase.

Phase A includes two key reviews:

• System Requirements Review (SRR): checks that high-level requirements for the mission will support the intended mission, and that they are complete and consistent.
• Mission or System Definition Review (MDR or SDR): checks the overall structure proposed for the system, and how requirements flow to major subsystems. Checks that initial measures of the design are reasonable. Also checks that management plans are in place.

The NASA Phase A maps to the second part of the concept development phase in the reference life cycle, along with concept and specification and preliminary design steps for the highest-level components in the system.

30.2.4 Phase B—Preliminary design and technology completion

Phase B continues the work from Phase A, completing a preliminary design and refining any new technology to the point where it is sufficiently mature to use in flight. This often involves building models and prototypes of parts of the system.

Phase B also involves developing the safety and security of the mission. The high-level design should incorporate designs for safety, security, and other critical mission success factors, and the design should be backed up by analysis showing why the design is sufficient. (See Chapter 43 for more on safety design.)

At the end of Phase B, the project should have a high-level design for the entire mission. That design should meet all the mission objectives, be technically feasible, and fit within cost and schedule available.

After Phase B, the agency allocates money to actually implement the system. The process can be complex and time-consuming, potentially involving legislative approval. The estimates for cost and schedule should be accurate enough that the project is unlikely to exceed them, which would require repeating the process to find more funding or time. This imposes limits on how much risk the project can carry going from Phase B to Phase C.

There is one key review in Phase B:

• Preliminary Design Review (PDR): checks that the high-level system design is complete and meets mission objectives, that any new technology needed is ready, and that safety and security requirements are met by the design.

The end of Phase B maps to a slice across the development phase in the reference life cycle. It includes the concept, specification, and preliminary design of the first two or three levels of components in the breakdown hierarchy (Section 11.3; Chapter 38). In general this might include the major spacecraft subsystems—payload, structure, propulsion, attitude control, and so on. The portion of the design step includes prototyping or modeling of components that pose technical risk, and the design may go to deeper levels of the breakdown hierarchy if needed to understand and address that risk.

The mission-level PDR follows reviews of the component-level preliminary designs.

30.2.5 Phase C—Final design and fabrication

This phase is when most of the development and production work is done. It involves designing, building, and verifying all the components in the system, to the point where they are ready to be assembled into the working spacecraft and ground systems.

Phase C is designed around the spacecraft being difficult and expensive to assemble, involving building large structures, using complex manufacturing tools, threading complex wiring harnesses through the structure, and putting large amounts of money at risk during the assembly. This leads to organizing the final assembly work to avoid as much risk as possible by ensuring that all the components are ready to assemble before committing to the final assembly steps.

During this phase, the team completes all of the designs and implementations of the system components, and verifies all of them. This usually includes producing engineering and qualification units of hardware components (Section 27.8) for testing, including destructive testing for some parts. It also usually includes integrating all of the engineering or qualification units and the corresponding software into a testing version of the entire spacecraft in order to verify the entire integrated system.

Verification in Phase C typically includes verifying the human interfaces in the system. Can an operations team use the ground systems to accurately control the spacecraft, using simulated telemetry showing the spacecraft in different conditions (including off-nominal conditions).

Phase C is typically divided into two parts: the first part for completing all the designs, and the second part for implementing and producing the components. The Critical Design Review separates the two parts, where all the designs are checked.

I have seen the Critical Design Review milestone cause confusion: how far should work progress before the CDR? What is the boundary between “design” and “implementation”? For hardware components, such as an electronics board, engineers work on the board design: the layout of the components and traces that will be fabricated. The NASA CDR definition ([NPR7123, Table G-7, pp. 113-4]) indicates that the CDR should include “integrated schematics” and “fabrication, assembly, integration, and test plans”, which would indicate that the board design is complete. That the document also indicates that the CDR and Production Readiness Review are often coupled lends credence to the interpretation that the CDR reviews the board designs.

If this same interpretation were applied to software, it would imply that the software would be essentially complete by CDR. Software source code is the equivalent of electronics board design: while it is thought of as implementation, it must be processed through a build system to produce the actual executable software, just as a board’s design file is used to manufacture the boards.

However, the NASA Systems Engineering Handbook states that the CDR for a software component should occur “prior to the start of coding of deliverable software products” [NASA16, Section 3.6, p. 29]. In other words, the documents appear to disagree, though NPR 7123.1 is presumed to have precedence.

Further, software is often developed iteratively, implementing one version after another, each version adding some amount of functionality over time. Some lower-level functionality is left as a mockup, perhaps not even fully designed, until some of the higher-level integrated functionality has been implemented and verified (the idea of integration-first development (! unknown reference XXX), done to reduce risk as quickly as possible). Software development best practice also has verification proceeding continuously throughout implementation, with feedback to the implementer as early as possible. This often implies having some of the hardware components built and available for testing the software before the software is completed.

An official answer to how a team should resolve the discrepancies and interpret the CDR for a NASA project will have to come from the relevant NASA authorities.

However, in practice, I have found that focusing on the review before implementation is more useful for components in the upper and middle levels of the breakdown hierarchy. For example, this might include the major components with a subsystem, such as power distribution or generation within the electrical power subsystem, or attitude control algorithms in the guidance, navigation, and control subsystem. Components at these levels realize the important relationships between components in the system structure (Section 12.2) and the way components work together to produce emergent properties (Section 12.4). Analyzing these designs allows one to check whether key system behaviors will be met, and that properties like safety or reliability are handled correctly. These are the properties that are difficult to change if the implementation is found during verification not to meet them. The design and implementation of low-level components should be reviewed, but as long as there is an obvious, low-risk approach for them their review need not block the design reviews of the system as a whole. This interpretation, of performing the critical design review before implementation, means that the team is then free to implement software components incrementally if that is the best approach for that part of the system.

There are three reviews in Phase C:

• Critical Design Review (CDR): a set of reviews of all the specifications and designs for the system, from the lowest-level component to the system as a whole. If a component’s design passes this review, it is ready to be produced. Each major component in the system—both flight and ground systems—has its own CDR, working from the bottom up until finally there is a CDR for the system as a whole.
• Production Readiness Review (PRR): for components that will be produced in some quantity (typically three or more), checking that the production procedures, tooling, and people are ready to build them. This is sometimes conducted alongside the CDR for the component.
• System Integration Review (SIR): checking that all of the components are on track to be assembled into the system, and that the procedures, tools, and people needed for assembly are ready. “Integration” in this review means “integration into the final assembly”, and not the more abstract integration that occurs during development and verification.

The NASA Phase C maps to completing the development phase, the acceptance phase (Section 27.9), and the system production phase (Section 28.1) in the reference life cycle.

The CDR milestone maps to a slice through the system and component development phases, at the end of the design step for most or all of the components. The PRR for a component is equivalent to a review at the end of the production unit development step (Section 27.8). Note that the reference life cycle has a manufacture and deployment check milestone in the acceptance phase; this applies when the entire system is manufactured together, rather than the model implied in the NASA life cycle where different hardware components go to production individually. Finally, the SIR is equivalent to the deployment readiness review that is at the beginning of the deployment phase (Section 28.3) in the reference life cycle.

30.2.6 Phase D—System assembly, integration, and test, launch and checkout

Phase D covers the work between the end of designing and building all the parts and having a spacecraft on orbit ready to begin its mission proper. This includes assembling the spacecraft and ground systems, and verifying that they work (and work together). The verification typically involves testing the assembled spacecraft in vacuum, under strong vibrations, and in thermal environments equivalent to what it is expected to handle in flight—but not testing beyond those levels, in ways that might damage the vehicle. After testing, the team proceeds onward to integrating the spacecraft with its launch vehicle, final preparations, launch, and starting operations on orbit. The team on the ground finally checks the spacecraft out before declaring it ready to begin its mission.

Some missions build a second copy of the spacecraft to be used on the ground for debugging issues with the one in flight and to test possible commands before sending them to the operational spacecraft. The duplicate is typically assembled in Phase D. It might use qualification units for hardware that were used for testing in Phase C, rather than flight-ready units.

There are several reviews in Phase D. All of them are final checks that some part of the mission is ready for taking an irrevocable step. These include:

• Operational Readiness Review (ORR): that mission operations is ready for the flight to start.
• Mission or Flight Readiness Review (MRR or FRR): that the flight parts of the mission are ready to go.
• Launch Readiness Review (LRR): that the launch vehicle and supporting systems, including the site and launch range, are ready.

The NASA Phase D maps directly to the deployment phase in the reference life cycle. It takes in manufactured components and procedures, assembles them into a working system, tests that it has been assembled properly, and starts it in operation. The milestones in the NASA Phase D are different from the deployment phase milestones mainly because they are specific to launching a spacecraft.

30.2.7 Phase E—Operations and sustainment

In this phase, the team operates the mission through its end.

There are two kinds of reviews that occur in Phase E:

• Critical Event Readiness Review (CERR): reviews before specific mission events where something could go wrong. This might be adjusting an orbit, or uploading software updates.
• Decommissioning Review (DR): reviewing the decision to end the mission.

Phase E is equivalent to the system operation (Section 28.5) and evolution (Section 28.7) phases in the reference life cycle. The Decommissioning Review is equivalent to the decision to retire the system at the beginning of the system retirement phase (Section 28.9).

30.2.8 Phase F—Close out

The final phase in the NASA life cycle involves retiring and disposing of the flight systems, retiring or releasing ground systems, archiving mission data, and closing out the project.

There is one review identified in the NASA life cycle:

• Disposal Readiness Review (DRR): determining that the mission is ready to command the disposal of the spacecraft.

This phase corresponds to the system retirement (Section 28.9) and project ending (Chapter 29) phases in the reference life cycle.

Part VII: Specifications

The first part of the life cycle: purpose, concept, and specification.

Chapter 31: System purpose

31.1 Introduction

• why one investigates and records purpose
  • avoid building something people don’t need
  • meet all the relevant needs
  • catch things they need that they don’t articulate right away, or that they imply
  • understand what market segment will be met, or how big the market is, for products that will be sold to multiple customers (interaction with marketing)

31.2 Investigating purpose

• investigating purpose
  • grounding in the users/stakeholders so assumptions can be caught
  • working out stakeholders
• how to gather information about purpose (tips)
  • interviews
  • active listening
  • feedback using models
  • play-acting
  • getting multiple viewpoints and combining and cross-checking

31.3 Recording purpose

• recording purpose
  • source documents (transcripts, notes) that are unfiltered
  • extracted and summarized user statements
  • organized objectives and constraints
• keeping purpose focused
  • how to decide whether to include some objective or not
  • market segmentation as part of the basis
• verifying purpose
  • mapping user statements to objectives and constraints, and showing them the objectives/constraints
  • use a mockup, prototype, or paper version
  • feed back concept when that is developed from the objectives and constraints

31.4 Changes of purpose

• detecting when users/stakeholders start talking about something new
• hooking into the change request process for explicit changes
• deciding whether to change part of the purpose
• estimating effects of a change

Chapter 32: System concept

32.1 Copied in

work to build a concept

• come up with lots of possible approaches
  • sketch out parts of system structure or concept of operations
• evaluate them as to whether they will help meet the need or not, and whether they are technically feasible (trade studies), sometimes combining best aspects of different approaches
• evaluation includes seeing if there is some available component, or some technology available, or some prior experience on which some top-level component can be based
• evaluation also includes back-of-the-envelope analyses of how difficult something will be to design and implement (time and resources, not uncertainty/risk), and how well it supports stakeholder needs (likely to meet performance or scale? fit with security and safety?)
• mapping out the flow to achieve some external behavior reveals gaps in internal structure
• along the way, track which stakeholder needs are being met and work out how to fill gaps
• often find that some part of the stakeholder needs worked out in the previous phase are insufficient or incomplete or inconsistent, causing some part of the purpose development to rewind to fill in the gap in purpose

XXX rewrite to reduce use of conops term

XXX harmonize stakeholders with list from earlier chapters

XXX harmonize concept contents with earlier chapters

XXX pull out document management section

XXX split purpose and purpose investigation out from concept

XXX discuss internal and external view, consistent with section 26.4 on concept development phase

32.2 Purpose of concept development

At the very beginning of a system development project, there is generally only a rough idea of what the system should be. The understanding of the system objectives is too vague to launch into development or writing tests right at the start.

The purpose of developing the initial system concept is to get a reasonably clear initial definition of what the system should be or do. The definition should accurately reflect what the customer needs (whether the customer is an actual customer or a representative of an expected customer). The definition need not be perfect; it will be revised as the project moves forward and the initial concept gets validated or the understanding of the customer’s needs improves.

Internal to the project, the initial concept provides the information that the team needs to begin working out the structure or architecture of the system and to begin writing the high-level system specification. In the absence of a well-structured concept, the development team cannot begin working out how to implement the system without taking risks that the design is wrong and will have to be redone.

Outside the project, concept development supports the relationship between the project and the customer. Clear agreement between the customer and the development team makes for efficient and less fraught development. Agreement on the concept can support writing a contract for development that protects both the development team and the customer from feature creep or extra costs.

The documentation of the concept will be used throughout the entire life of system development and operation. The concept is the record of the big picture for the system. While it gathers the information needed to guide system design, the big picture is also important to new people joining the team who need to learn about the system they will be working on. The concept also serves management as a definition of the goal that system development is trying to reach, allowing the team to check from time to time whether they are designing and building the right thing. In that way a good definition of the concept is essential to being able to validate the system design.

32.3 Concept development work

The concept development work in a project seeks to establish a clear statement of what the system should be and do. At the end of the work, there should be a record of the customer’s objectives for the system, a concept of operations that embodies those objectives and explains what the system is, and review and approval from both the customer and the project leadership.

The core of this is to determine and record what the customer wants. This is rarely an easy task: there may or may not be a well-defined customer, and the customer may or may not be able to articulate what they need and what they want.

Start with working out who the customer is. Some projects are customer driven, meaning that there is a customer for the system and the project is working in response to that customer’s needs. A customer who contracts with a development team to build a system for them is the simplest example of this kind of project. Other projects are RFP driven, meaning that there is a customer but they are asking one or more teams to propose a system design before committing to funding development. These projects differ from customer driven projects in that the customer provides a request for proposal (RFP) that should contain all the information needed about what the customer wants. (In practice the RFP is rarely sufficient.) Still other projects are visionary, meaning that there is no specific customer driving what the system should do, but instead the system is being designed in the expectation that there will be customers for the system in the future. This kind of project includes innovative systems that are expected to help create a market for themselves.

Every project should document who their customer is and how they work. Where there is an actual customer, this should include information about how they make decisions about systems, who the decision-makers and influencers are, and any contacts that might be able to provide background information or advice about the customer. For visionary projects, common practice is to develop a profile of one or more hypothetical customers, including how they are expected to decide whether to acquire a system and where information can be found to characterize these potential customers.

A document recording what the customer wants is the first major work product in the concept development phase. This document serves as the primary record of what the customer has asked for. It is used in later work to check whether the interpretation that the development team puts on what they think they have heard from the customer is accurate. It is vital that the customer objectives document be free of biases that the project team brings, because this document is used to detect when the team have applied their biases.

The customer objectives document, therefore, should be written using the customer’s own language, and should only be a summary of what the customer has said. One way to achieve this is to collect primary source material from the customer—such as notes or recordings from meetings, a request for proposals, or externally-sourced market analyses—and then write a summary of their contents. If possible, the summary document should include references to the primary sources so that someone can check the summary for accuracy. Writing the objectives document as prose, or as a bulleted list, is reasonable. The customer objectives document should not be organized into formal requirements; that step comes later and formal requirements should derive from the customer objectives document.

Where possible, the customer objectives document should be shared with the customer (or representative potential customers) to validate that the document is accurate. It is normal for there to be many iterations with the customer to get the details right; indeed, that is the point of gathering the customer objectives.

The next major document, or collection of documents, records other constraints on the system’s design. This includes things like

• Regulation to which the system or the development team must adhere
• Company or organization standards and objectives, including business objectives, safety objectives, and security objectives
• Customer standards that apply to the project

This document of other constraints should reference the source material for each kind of constraint.

With the customer definition, customer objectives, and constraints documented, the team can develop a concept of operations, or CONOPS. This document records a simple model of how the system can be structured and operate at a high level. The contents of the CONOPS should be limited to how the system will interact with things outside itself, whether that is the function and structure that the customer will see, the interactions with other systems, or its interactions with regulatory organizations. The CONOPS should be fairly brief, and diagrams are helpful. The CONOPS should not become a specification of the system’s design; again, the specification and design are derived from the CONOPS. The CONOPS document is often written in the language that the development team uses. It is common to use graphical notation standards for some elements where appropriate.

The contents of the CONOPS should derive from the customer objectives and other constraints. A good CONOPS document will include references to those sources to show why the concept is structured the way it is.

At the end of the concept development phase, the team will have gathered information about what the system needs to be, both to satisfy the customer and to satisfy other stakeholders, and created a conceptual version of the system’s functions in the CONOPS.

The CONOPS should be shared with the customer for their review and approval. Because the CONOPS document is written for the development team using the team’s language, it is often necessary to interpret the contents of the CONOPS to the customer. The customer should validate that the functions included in the CONOPS cover everything they are expecting. The development team’s management should also review and approve the concept material to validate that it meets organization policies and regulatory needs.

Sidebar: Stages of working out the system concept

Some people have asked: what is the difference between the customer objectives, concept of operations, requirements, and design? Why have all these different documents? Why not begin by recording requirements?

We view the process of working out the system’s concept as a multistage flow, starting with a vague idea and ending in a concrete specification, including requirements, for what the system should do. The flow is:

The customer objectives and concept of operations documents deal only with the concept of the system—the abstract general functions and purpose. Most importantly, they should be understandable by the customers, who look at them to see if their ideas and needs are being reflected in the system that will be designed.

Take requirements, for example. The content of every top-level requirement that goes into the system specification should be present in the customer objectives and concept of operations, but the information should be recorded informally, accompanied with explanation of the context and purpose of each function. These will be translated into requirements in the system specification, which have a strict format and are typically recorded in a database that is not easy for customers who are not also trained in systems engineering to follow.

32.4 When the concept is complete

The goal for the initial concept development phase is to have a clear but informal understanding of what the customer wants, what constraints other stakeholders place on the system, and agreement from all the parties that the documented understanding is correct, so that the team can move on to formalizing the design of the system.

Put another way, the system design efforts depend on knowing what the system is supposed to do. The amount of effort put into system design should be low until the team has confidence that they understand what the system design should do.

Specifically:

• The team has confidence that it has a clear and complete idea of who the customer is and what the customer’s objectives are
• The team has confidence that it has an accurate understanding of other constraints that will be placed on the system
• The team has developed a concept of operations that plausibly addresses the customer objectives
• The customer (or their proxy) can confirm that the team has understood their needs
• The team’s management can confirm that the system described in the concept of operations is likely both to be feasible and to meet the organization’s business objectives

32.5 Evolving the concept

The concept for the system will change over time. This can happen in the early stages of a project, when one is still working with the customer for the first time to understand their objectives. It also happens later: when the customer’s needs change, when the team realizes that they have misinterpreted the customer’s objectives, or when regulation changes.

In general terms, while working to develop the system concept for the first time, any changes can be made as needed. At some point, the initial versions of the concept documents will be “done”, reviewed, and approved. The documents are then baselined: marked as stable, meaning that people can use the information in the baselined concept documents to develop system architecture and specifications without worrying that the documents will be shifting on them all the time.

After the concept has been baselined, the team must follow a more careful process to make changes. First one identifies what has changed: a change in the customer objectives, or in constraints such as regulations. Next one analyzes the change to determine where the concept documents need to be revised. The revised document versions exist in parallel with the baseline version, but only the baseline version is official until the revised documents are reviewed, approved, and marked as the new baseline.

When the baseline version is updated, changes may need to propagate to other documents. For example, the system architecture and specifications derive from the concept documents. A change that adds a function to the system concept in the CONOPS document will induce changes to the architecture (possibly adding new components) and specifications (adding functional requirements to some components‘ specifications). If the change happens late in the project, when parts of the system have been implemented and verified, the change may propagate all the way to updated software, hardware designs, and test cases.

This explanation of the process simplifies the steps somewhat: individual documents are versioned or baselined. A change to the customer objectives results in an updated version of the objectives document. This leads to an updated version of the CONOPS, which in turn can lead to a need to review and approve the updated CONOPS, as shown in the diagram above. The new versions of the documents should not be baselined until they have collectively been reviewed and approved. They should be baselined together, so that the official, stable versions of all the documents remain consistent with each other. This means that updated versions should remain work in progress until the review step has been completed.

32.6 Concept development for different kinds of projects

Concept development is all about knowing what the customer wants. But who is the customer? How does one learn what they want? There are multiple answers to these questions.

We can divide projects into three general groups:

• Customer-driven, where there is a specific customer who wants the system. One can learn what this customer wants by talking to them. This is typical of internal customers, or customers who have an ongoing contractual relationship with the development organization.
• RFP-driven, where there is a specific customer. This customer is following an acquisition process where they provide a request for proposals (RFP), and the team must respond with a proposal for developing the system. The customer is often constrained in how they can communicate about their needs in order to be fair to all proposers. This is typical of government contracting.
• Visionary, when there is not a specific customer yet, and possibly not even a defined market for the system. This is typical of commercial products that will be marketed to many customers once it is created.

32.6.1 For customer-driven projects

Customer-driven projects are those that have a specific customer whose needs or desires drive what the system should do. The development team is focused on satisfying this specific organization.

In these projects, the development team can communicate with the customer to learn how the customer works and what their needs are. Ideally, the customer will be involved all through system development, so that the team can check what they are building directly with the customer.

The Agile Development approach to software development grew out of customer-driven projects. The Agile approach advocates for the customer being continuously involved, including helping to prioritize work in each development sprint. This kind of direct involvement is only possible when the development team can interact constantly with the customer. Note that we do not advocate dogmatic Agile Development (nor do we think many projects actually use it); we will discuss this more in chapters on validation and management.

Some customers will begin working with the team with only a general concept in mind. The team will need to draw out from the customer what that concept means, and explore all the corner cases with them.

Other customers will bring a partially-developed concept of what they want. In these cases, the team must, first, ensure that they properly understand what the customer is saying, and second, explore the concept with the customer to find any missing information.

Working with a customer on the concept is full of pitfalls. The most serious is that the team will interpret what the customer is saying in a way that the customer does not actually mean. The development team can bring their own interpretations to understanding what the customer says; the result can be a system design that doesn’t meet the customer’s actual needs. The team should use communication techniques that allow them to validate their understanding of what the customer is trying to say, such as active listening ! Unknown link ref methods. The key ideas in these techniques are

• To read back in paraphrase to the customer what the team has heard
• To ask clarifying questions when someone thinks there may be multiple interpretations of a statement, and
• To ask open-ended questions rather than prompting the customer for a particular answer.

There are many references available on these techniques.

The concept needs to include everything that the customer actually wants. Most commonly, the customer will be thinking about the most important functions they need but will not be considering all of the other functions that are needed to make the important functions work.

The team needs to work with the customer to elicit these other functions or use cases. While there is no recipe for finding all these other use cases, we have found that there are some questions that help ferret them out.

• For use cases that the customer is thinking about, how does the system get set up to be ready to start that use case? For example, for an aircraft autopilot, what conditions must be true for it to operate? How does a pilot set it up and engage it?
• For each step of the use cases, what happens if something goes wrong? Or if a user changes their mind? For a banking system, what happens if a user is part way through setting up a transaction and realizes they entered the wrong amount in an earlier step? What happens when an aircraft’s engine stops working during a takeoff roll?
• How does the system go through its life cycle? If multiple instances will be manufactured, how are parts supplied? How is it manufactured and tested? How is it disposed of?
• What maintenance will be required? Are there mechanical parts that require regular inspection and replacement? Are there information components with logs that will fill up?
• If the system has users who are authenticated, how are user accounts managed? What are the assumptions about the users? How are they vetted or trained? For example, what training or licensing does a pilot require for a new aircraft design?
• Who are the regulators or other responsible organizations, such as courts or industry trade groups, who may be called upon to audit the system’s operation?

32.6.2 For RFP-driven projects

An RFP-driven project is one where a customer is asking for proposals from development teams about how they will design and build a system. The customer is usually asking for multiple, competing teams; the customer will choose one or more teams for a contract to build the system.

The customer writes a request for proposals (RFP) document that defines both the characteristics of the desired system and how the customer expects to judge between multiple competing proposals, if there are any. The RFP should thus document the customer’s objectives. In many competitive acquisition cases, the RFP must be the only official source that a team can have so that all proposing teams work from the same information—thus treating all teams equally.

When deciding to respond to an RFP, the team must learn what acquisition rules the (potential) customer is using in order to determine what restrictions to follow when communicating with the client. The team must also learn how the customer makes decisions, including who makes the decisions, who influences the decisions, and how the decision will be made. When responding to a commercial RFP, this can be easy: there is a contact who sends out the RFP and who can answer questions as needed, there is someone they work for who reviews and decides whether to accept a proposal or not, and the decision is based on what the decision-maker thinks meets their needs at the best price. For a US Government agency RFP, on the other hand, the decision process is defined by Federal Acquisition Regulations and by the agency’s supplemental rules. There are formal processes for submitting questions; there is typically a defined scoring and weighting system that a formal review team must use to rate each proposal.

The information gathered about how the customer communicates and makes decisions should be included in the Customer Definition document.

When the customer is doing a competitive acquisition, the team also needs to gather information on the other teams that may be choosing to submit a proposal. This information helps shape the proposed design and the proposal itself to make them look better than the competition to the customer. This can include relative strengths and weaknesses of the other teams, such as whether this team has proprietary technology that will do a better job for the customer (a weakness of the other team), or whether the other team has more flexibility in pricing (which might be a strength of the other team). This information should be gathered into a Competition document.

In practice RFPs are rarely complete or unambiguous. This is because they are written only by the customer, and there is little opportunity for dialog so that the customer can get alternative perspectives and check that their work is clear and complete. When it is possible, the team should engage in the kind of dialog with the customer that they would in a customer-driven project in order to confirm their understanding of what the RFP says and to flesh out the request to include a more complete picture of what the customer actually needs. When this is not possible, the team should find people who can accurately represent the customer’s way of thinking and needs, such as people who have a similar position in a different organization in the same industry, or someone who has worked closely with the customer in the past and knows the business or people involved.

Whether one can get clarifying information or not, the concept documents should include documentation on where the team has made assumptions or interpretations of the RFP source material. These points are matters where there is greater than usual risk that the team’s assumption does not match what the customer is thinking. This means that there is a higher than usual risk that the concept or design that the team proposes will be interpreted differently than what the team means—and so it is worth putting extra effort into making those parts of the proposed concept or design as clear as possible.

There are two end results of the process for responding to an RFP: first, a decision whether to complete and submit a proposal, and second, submitting a proposal if the first decision is positive.

The decision about whether to submit a proposal or not depends on

• Whether the team has the resources to pursue the work
• Whether the team has a reasonable chance of winning a contract, and
• Whether building the system would be worthwhile.

Determining whether the team has resources requires estimating the resources needed. For the first steps of concept development, this may be small, perhaps one or a handful of people to gather information and to get an initial understanding of what the customer wants. As the work progresses, more resources will be needed—to gather more information, to do concept development, to gather competitive market data. At each step of the process, it will become clearer how many people or other resources are needed for the next step of developing the proposal. At the same time, the team must be able to estimate how much resource will be needed to build the system if they win a contract. This will be unknown to start, but as the system concept and architecture work move forward the estimates will improve. The team must develop the architecture enough to be able to determine prices to charge the customer and to be able to determine if the team will have the capacity to do the work. These analyses grow out of the concept of operations and later architecture documents.

Determining whether the team has a reasonable chance of winning is a combination of knowing how the customer will judge proposals, how strong other teams are likely to be, and how well this team can satisfy the customer. This information is gathered in the Customer Definition document, the Competition document, and in how the Concept of Operations and architecture respond to the customer’s objectives.

Finally, determining whether building the system can be worthwhile depends on knowing what the team’s organization values. Does the organization require a particular profit margin? Is there a minimum or maximum contract price that is considered “interesting”? Does the system fit within the organization’s business strategy? These kinds of questions are captured in the Business objectives document, and analyses use customer objectives, CONOPS, and architecture documents to develop an answer to them.

Developing proposals is a complex specialty, and much has been written about it. We refer the reader to ! Unknown link ref for further reading.

32.6.3 For visionary projects

A visionary project, as we are using the term, is one where the system being designed and built is not for a specific, existing customer. Instead, the system might be marketed to several potential customers down the line, or the system might be part of a strategy to change an existing market or create a new one, thus creating new customers who may not even exist yet.

Consider building a new commercial passenger transport aircraft. The air transportation system is mature, and so one can name who buys these aircraft: airlines, aircraft leasing companies that provide the aircraft to airlines, businesses using aircraft for private transportation, and government organizations that fly passenger aircraft. No aircraft company in recent decades has built a new large passenger aircraft to be sold only to a single customer; instead, the companies work out the needs of many potential customers and design an aircraft that will be good for many of those customers. Since airlines come and go often relative to the lifetime of an aircraft design, many of the potential airline customers do not yet exist when the aircraft company has to decide on the capabilities of the new aircraft. This is a case where the market exists but there is not a single customer to satisfy.

In contrast, consider the first generation of global satellite data and telephony networks (such as Iridium and Globalstar). When they were being designed, there was no mass market of ground-to-space mobile communications. These companies, and others that did not end up deploying their networks, had to work out who their potential customers might be and what they might need. Indeed, all of these first generation providers went bankrupt at some point as they developed both their network systems and at the same time built up a subscriber base. This is an example of a project that was creating a new market.

In both these cases, there is not a single definition of a customer. Instead, the team must determine the market—the set of customers—who might want the system. The team looks for the set of features or capabilities that will satisfy a large enough market to be worth supporting. The plan will often be to start with a small market segment and grow over time by adding capabilities to satisfy more people, while having learned more about the first set of customers and gaining some revenue to help fund growth.

All this information will need to be collected from a number of sources, including market analysts, surveys of potential customers, and the experience of people who have worked in related industries. Finding people or organizations who can act as a proxy for a class of possible customers is helpful. It is important to gather from multiple sources in order to cross-check the information and to account for sampling bias that can happen if information comes from only one perspective.

The information about the target market segment(s) will change regularly over the course of the project as customers come and go, or as new opportunities appear. This means that the design and implementation of the system will likely need to adjust as time goes by. This also means that the team needs to continue to survey the market and talk to potential customers.

At the same time, it is a rare project that can successfully chase arbitrarily changing customer objectives. The design and implementation team needs enough stability that they can complete a version of the system. Marketing and sales teams need stability so that they know what they can actually sell to a customer. The stable version of the customer objectives should be baselined (see section on configuration management below). Changes to the baseline should occur only periodically, when the team decides that either there is a change in the understanding about customers that is vital to reflect in the design of the system right away, even at the cost of delaying the system being ready for use, or when there is a change that does not delay or significantly change the system being designed and built right now.

The idea of a minimum viable product (MVP) is fashionable in recent years. The general approach is to create the simplest system that will meet the needs of just a few customers, put the team’s focus on building up that first version, then plan on adding capabilities as time goes by to make the product attractive to more customers. This is an example of planning how to handle changes in understanding what customers want.

Visionary projects can expect that there will be competition with other teams’ products. Indeed, customer choice is a fundamental precept of the Western market system, and often required by regulation. A team should develop a record of what their competition might be, whether that is another organization offering a similar product (as happens with large passenger aircraft), or whether a customer could meet their needs a different way, or whether customers will choose no to buy a new product and live without its benefits (which is common with new technology trying to create a new market). The team should also build up an analysis of what sets this team’s system apart from alternatives—why a customer would choose this system over other options. Maintaining the Competition document with this information will help the team make decisions about changes to the customer objectives or business objectives around which the team is designing the system.

32.7 Artifacts

The concept development phase involves artifacts such as the customer objectives and concept of operations. We can now define each of these artifacts, but first we will address document management as a necessary supporting capability.

32.7.1 Document/configuration management

Each of the artifacts worked on and produced in the concept development effort should be placed under document management. The document management system should provide:

• A place to store each of the document, in a way that is easy for everyone who needs to find the documents
• The ability to store both baselined and working versions of a document, and a way to turn a working version into a baselined version
• A way to distinguished between versions of a document that represent work in progress and versions that are stable

A project should establish a document management system early in the initial concept phase. The concept will be represented in the artifacts listed below, and when these artifacts are reviewed and approved, a baselined version of each should be available in the repository.

32.7.2 Customer definition

Form. The customer definition document is generally a prose document; it does not need structuring the way objectives or requirements do. Some organizations may have customer relationship management tools that will capture some of the content defined below.

32.7.3 Customer objectives

Purpose. The customer objectives document is a record of what the customer wants out of a system. This document is a summary of what they have said they want and what has been drawn out from them in discussion or research. The objectives document is the source of all the rest of the artifacts developed for the system.

The objectives should record

• Who or what uses the system
• How the system behaves toward each of those users

The customer objectives document should be as close to the customer’s words and organization as possible. The document is a summary of the customer’s wants and needs. It is used as a proxy for the customer throughout system development, rather than having every developer talk directly to the customer to check their specification or design.

Other information must be kept out of the customer objectives document. The other information is captured separately in things like the regulatory and business objectives documents, which we discuss next. We have seen organizations include their business objectives, such as profitability, in the list of customer objectives. We have also seen teams include internal technical objectives like being able to reuse parts of existing designs included. Doing so creates confusion: is an objective in the document actually something the customer wants, or is this something the customer doesn’t care about? There will come times in the development process when hard decisions must be made about some part of the system design; at those moments, the team must be clear about what is actually a customer need and what is an internal need. If a customer need can’t be met reasonably, then the team needs to talk to the customer to resolve the issue. If an internal business or technical objective is proving hard to meet, the decision should be handled internally and the customer should not be involved—they don’t care or know about the issue.

32.7.4 Regulation and process definition

While the system being built needs to meet the customer’s needs, there are other stakeholders whose needs must be addressed as well. These are, broadly, internal objectives of the development organization and external objectives of third parties, such as regulatory bodies. Some of these objectives will define capabilities that the system must have. Other objectives provide constraints on how the system can function or be implemented, without defining specific capabilities.

32.7.4.1 Regulatory objectives

Many kinds of systems are subject to regulation. Some systems require licensing or certification, to prove that they meet regulations; others only need to be able to show compliance on demand.

These regulations pose constraints on the design of the system. Some are simple: aircraft emergency exits must be marked in particular locations. Some are complex: the crew of an aircraft must be able to properly determine what is happening with an aircraft even when there are complex failure situations—which involves human factors as well as the design of aircraft sensing systems.

A system will not be able to be put into operation unless it can meet these regulations. This means that the regulations must be incorporated into the design, just as the functional desires of the customer must be. One cannot do this unless one knows what the regulatory constraints are, and so one must search out and document all the regulations that apply.

The regulatory objectives document should at minimum list the source regulatory documents that apply to the system. Before design validation is complete, the information in the regulatory objectives document must translate into a detailed collection of requirements against which the system can be checked.

It is often necessary to involve either experts in the regulation of a particular industry or the regulatory agencies themselves to properly gather all of the regulations that apply.

32.7.4.2 Third party or service provider objectives

No systems operate in isolation. Instead, they operate within the context of a larger system of people, businesses, and organizations. This might include:

• Other systems that the customer already operates
• Services that the system will interact with directly, such as for maintenance or communications
• Services that the system will interact with indirectly, such as finance organizations that provide operating capital for the customer’s day-to-day operations
• Services or organizations that maintain the environment in which the system operates, such as a data center, an airport, or roads
• Part or component suppliers that provide components that will be used inside the system

The interactions and dependencies within this larger system also create constraints on how the system being designed must function. It is important to identify each of these organizations or systems, document how the system will interact with them, and then document the more specific objectives that are involved in working with them.

This information should be collected into one or more documents that record, first, the structure of the larger system and its interfaces with the system being designed; and second, the sources of constraints or objectives for each interface.

Information about the ecosystem in which the system will operate is likely to change frequently over the course of developing a system, especially for visionary projects. This means that it is important to update information about these objectives, and when it changes, flow those changes down into the system design.

32.7.4.3 Business objectives

An organization that is devoting resources to build a system must be able to obtain those resources. At the minimum, the organization must be able to hire and pay the people who design and build the system; it must be able to pay for the tools and prototypes it uses; it must be able to pay people to gather customer objectives and work with regulators and all the hundred other tasks involved.

Most organizations are also building an ongoing business, not just coming together long enough to build one system and then disbanding. Sustaining a business requires obtaining funding, getting sufficient return on the work the organization does in order to fund continuing work, and building capabilities that allow the organization to keep building or maintaining systems into the future.

All these imply that an organization needs to have a business strategy, which leads to business objectives. The organization may have a strategy of developing a product line that serves a wide variety of customers. This might translate into an objective to build a simple initial system product that is able to generate X revenue, and that can be extended over time to address the needs of more customers.

Many organizations develop these objectives at the executive level but do not feed the information downward explicitly to the team who must design a system. This is a problem because the design team knows that such objectives exist but don’t necessarily know exactly what they are, and thus can’t make accurate design judgments. We have seen, over and over, questions in a design team like “should we design this board with extra capability now, or design the minimal board and replace it later?” These have often led to arguments because the design team did not have the information needed to make a choice between a higher up-front investment cost for extra capability or incurring cost later in a redesigned board.

There are many different kinds of business objectives to document.

Some objectives are easily quantifiable:

• The budget for developing the system
• Deadlines for delivering the system and intermediate decision milestones

There are general business case objectives:

• The expected profitability of the system over time
  • The profitability of system development, if appropriate, or whether the development is purely an expense
  • How long it should take to recover development expenses
  • How profit or return on investment should evolve over time
• How the scale of the system should evolve over time
• How the size of the addressable market should evolve over time

Finally there are business strategy objectives:

• Is this project intended to build a single iteration of the system for a single customer? Or is it intended to be the base for ongoing development of an incrementally improving system?
• Is this project intended to produce other things of value besides the system itself, such as developing in-house capabilities in some area, or building technology or designs that can be reused (at low expense) in future projects?

These business objectives change continuously. When there is a proposal to change the objectives, the team must follow a disciplined process to determine what the effects of the change might be. This involves tracking down how the change will affect technical requirements and designs, which in turn affects whether the changes will affect the system’s ability to satisfy customer, regulatory, safety, or security needs. Changes to the design will also affect development cost and the time required to bring the system to operation. Sometimes a change to business objectives will make sense: changing the rate at which the system should scale up after the initial operational version may not affect the development time much but will increase customer satisfaction. Other times a change will have negative consequences: setting the goals for the size of the addressable market too high too early may require a higher development budget and longer development time than is available. Making a well-informed decision about these changes is only possible if the team can determine what the effects of a potential change in business objectives are.

32.7.4.4 Safety objectives

Safety is the condition that a system, when operated in the intended way, does not produce too many events that cause harm. There are four parts to this statement:

1. The operational system (its design and implementation)
2. The way it is intended to be operated (the intended functions and environment)
3. Produce too many events
4. Events that cause harm

In the end, a system must be shown to be safe by showing that the rate at which it causes harm is below a threshold. The process of designing a system to be safe is well known to be a difficult task, and there are many books and standards that try to give guidance on how to do so. As the system is designed, it must be evaluated to show the likely rates at which harmful events will occur.

This is a complex topic, and later chapters will address the design and analysis of safe systems. For now we focus on safety objectives.

Performing these evaluations requires defining what kinds of harms are to be measured, along with the acceptable rates at which they occur. There is no possible way to justify that a system is “safe” or “unsafe” without defining the harms they refer to.

A project, therefore, must define and document its high-level safety objectives in terms of the harms and the acceptable rates of those harms occurring. This is the safety objectives document.

Some industries have conventional definitions of harm and rates. The automotive industry has adopted a scale of zero to three for “severity” in the ISO 26262 standard ! Unknown link ref, focused entirely on injury to persons. Severity 0 is no injuries, 1 is light to moderate injuries, 2 is severe injuries with survival probable, and 3 is severe or fatal injuries. The aviation industry has defined a five-level scheme in the ARP 4754 standard ! Unknown link ref, ranging from minor (slight increase in crew workload or minor passenger inconvenience) through hazardous (serious or fatal injuries among passengers) and catastrophic (many deaths, loss of aircraft).

These two standards differ in two respects. They consider different ranges of harm: ISO 26262 has any severe or fatal injury as its highest category, while ARP 4754 considers the distinction between fatal injury and mass fatality. They also consider different kinds of harms: ISO 26262 only considers injury to persons, while ARP 4754 considers effects on the crew’s ability to control the aircraft and damage to the aircraft.

These point to deficiencies in the standards, and to the reason why a project should define its safety objectives more carefully. There are many harmful incidents that these standards do not address, such as damage to property, economic harm, or damage or injury to non-person cargo. Consider an incident involving a truck that damages an overpass, but does not injure anyone directly. The cost of repairing or replacing the bridge can run to several millions of dollars; the economic impact on the community of not being able to use the bridge can be equally high. In addition, depending on the industry, the range of severity in these standards can also be too limited: they do not account for harms that spread beyond the people and vehicles immediately involved in an incident. The use of aircraft as missiles in the 9/11 attacks showed how an aircraft safety incident can result in mass casualties or worse.

In addition to defining the harms that system design will consider, the safety objectives document sets targets for how often those harms can occur. Guidance issued for commuter aircraft ! Unknown link ref, for example, gives a maximum allowed rate of incidents per flight hour: XXX

Minor 10-3 Major 10-5 Hazardous 10-7 Catastrophic 10-9

The safety objectives document should define a maximum rate and the time interval over which that rate applies for each category of harm.

Some organizations may choose to say that the system they build should allow zero safety incidents above a certain level. This is possible only if the system can be guaranteed never to perform operations that could induce such serious events. For example, an aircraft can be guaranteed never to cause catastrophic harm, involving multiple fatalities—but only if the aircraft has a maximum weight of a few tens of kilograms, a low maximum speed before it disintegrates in the air, can only carry a single person, and so on. No transport aircraft (more than 19 seats or maximum takeoff weight greater than 19,000 lbs) that actually flies can ever have a zero rate of catastrophic harm. Similarly, many weapons systems can never have a zero rate of mass casualty harms simply because of the energy they carry. In most cases, as the conventional wisdom goes, the only way to get a system to have a zero rate of harm is not to build the system.

Safety objectives, like customer, business, or regulatory objectives, are sources that lead to the concept of operations and top-level system specifications.

Defining precise safety objectives early in a project is required for building a safe system. We have observed many projects that made aspirational statements about “safety being a first priority”. In every single instance where the definition stopped at that statement, the team designed an obviously unsafe system—often because, in the absence of an objective standard, each person took steps they thought would be safe but in aggregate the design missed even basic scenarios that resulted in hazards. Further, the absence of an objective meant that no one could perform an objective analysis of a design to determine whether it was good enough.

32.7.4.5 Security objectives

The security objectives document provides guidance for how the system should be designed and validated to ensure that it can handle a reasonable range of attacks.

Security objectives are similar to safety objectives: they define a set of harms resulting from security incidents that the system must work to avoid or contain. Unlike safety objectives, however, security incidents occur as the result of malicious, intentional actions rather than as a result of failures, accidents, or design flaws. Like safety objectives, the security objectives document names the harms that the system should avoid. It cannot, however, generally specify maximum acceptable incident rates because the rate at which attacks occur is something that attackers can deliberately control.

The approach to defining security objectives, then, is to name threat actors and the harms they can cause. A threat actor is a person or organization that can choose to initiate an attack on the system, such as a hacker, a criminal organization, or a hostile nation state. Each threat actor can be characterized by their motivations (a criminal organization for financial gain, a nation state to disable defense-relevant capabilities). The harms the threat actors can cause include disclosure of confidential information, interruption of business, death of persons, financial loss, or theft of goods. The list of harms includes every kind of harm addressed as a safety concern, plus harms that do not involve damage or injury but do involve loss of value or information.

The system must then be designed to address the different harms that different threat actors might pose. The resulting design can be analyzed to determine whether the threats are sufficiently addressed. The built system can be tested to verify that key defensive features are working as intended.

The definition of “sufficiently addressed” remains subjective. Some security analysis techniques have rationales for assigning weights to different threats. For those analyses, ensuring that all high- and medium-priority threats have been mitigated might be sufficient.

There are many standards related to security, and depending on the industry and geographic region compliance with some standards may be mandatory. These may define security objectives that a system must meet for regulatory or business acceptance. This information should be documented in the regulatory objectives document, and information about threat actors or harms should flow from the regulatory objectives document into the security objectives document.

32.7.5 Concept of operations

Purpose. The concept of operations (CONOPS) document is the systems team’s response to the customer’s objectives. It collects in one place a description of how the system will work, from the point of view of the people who will use it. Whereas the customer objectives come from the customer and should record their needs from the customer’s point of view, the CONOPS shows how a system could behave in a way that meets the customer needs. The CONOPS is written from the point of view of the system.

The CONOPS document organizes the ideas about how the system will behave. In doing so it gives a model for understanding how the system’s functions can be organized and how different behaviors relate to each other. It does not aim to provide every detail about the behavior; its value is in documenting the big picture.

The CONOPS document has three primary purposes:

1. To collect a description of the system’s behavior in one place, which can then be used as the source to develop the system’s specification and design,
2. To provide a description of the big picture for how the system will work to people on the team, especially people joining the team, and
3. To feed back to a customer the system’s team’s understanding of what the customer has asked for, in order to check that understanding.

The document is typically written as a narrative, and not as formal requirements or detailed behavioral models. Its value is in its explanation, not its precision. It is an explanation of how the system might function, without reference to how the system can be implemented to achieve those functions. The details of operations, as well as the implementation, are recorded in documents that derive from the CONOPS. It should, however, expose the users, features, functions, states, and use cases that model what the customers’ objectives mean.

The CONOPS should include functions that are implicit in the customer’s objectives. For example, the document should cover the system’s entire life cycle, from deployment or initial startup through shutdown and disposal. The document should cover major faults that might occur, and the system’s behavior when those occur. For a spacecraft, for example, it should include recovery modes that allow the spacecraft (perhaps under ground control) to re-establish normal operation after a fault. It should include not just the technical core of the system, but also how the human or organizational elements that use the system behave. For an automobile, the CONOPS should not just say that there is a driver, but include expectations like the driver being trained and licensed. For an airline, the CONOPS should include the airline’s safety management system and how that interacts with an aircraft or maintenance technician.

To produce the concept, the systems team reviews and understands all the objectives documents already described, then follows a process to extract from those objectives a model of how a system could behave. The process of analyzing all the objectives will almost always reveal things that the customer or others have not addressed—customers often focus on the main operational behaviors, for example, and don’t address how to deploy or dispose of the system. The systems team needs to find these gaps and address them. Where possible, the systems team should work with the customer to check whether the customer in fact has expectations about these topics before committing to a concept.

32.7.6 Reviews and approval

Purpose. The initial concept development phase gathers and organizes information about what a system should be and how it should behave. It gathers this information from many sources: the customer (if there is one), third parties that impose constraints, the developing organization’s policies and standards.

The concept leads, in turn, to all of the technical artifacts that make up the system and its design: the specifications, designs, analyses, implementations, and so on. Those technical artifacts can only be as good as the concept from which they are developed, so checking that the concept is accurate and complete is vital to producing a good system.

The team is ready to proceed to system specification when two conditions hold:

• The team is confident that the system concept is complete enough and that it accurately reflects customer objectives
• The team’s organization is confident that the team can do the work. This means that the organization has the resources and skills to do the project; that the project is in line with the organization’s business strategy; and that the project is well enough defined to be confident in these evaluations.

32.7.7 Proposal (RFP-driven projects)

Purpose. A proposal, in the sense meant here, is a document that is sent to a potential customer in response to a request for proposal (RFP) that the potential customer has issued.

A proposal needs to make four cases to the customer:

• That the team has a technical solution that meets the customer’s needs
• That the team has the capability to actually produce the system
• That the price for the system will be acceptable
• That the team has a better offering than its competitors, if any

The proposal derives from the work done during concept development, but usually also must include initial system specification and design work. This initial technical work is needed both to be able to explain to the customer what they would be getting if they choose this team to develop the system, and to generate a reasonable price for building the system.

Many processes and guidelines for proposal development have been published over the years, and we refer the reader to that large body of literature for details.

32.7.8 Competition documentation

Chapter 33: Specifications

33.1 Purpose

Specification is about recording how a component (or system) should behave or the structure that the component should present. It only documents how the component appears from the outside, as a black box; it does not specify how the component achieves these ends. A specification derives from the less-formal concept for the system or component.

XXX address specification vs requirement

XXX make sure this ties into the broader flow of phases

A specification provides a simplified and abstract view of a component. This abstract view allows one to reason about how the component will work with other components. Without the abstract view, one would have to analyze the details of a component’s implementation to determine whether it will interact properly with another. While that is possible, the work of figuring out how the component will behave only serves to reconstruct design information that was originally worked out when designing the component. The reconstructed information will not necessarily match the information used during design, and the effort is wasteful.

A good specification records the intent and assumptions that went into working out what the component is supposed to do. This information helps the component’s implementer and designer to check that they understand what they need to build, and to check that the specification matches the intent. These assumptions also help people understand how a component might need to change when part of the system is redesigned—to add a new feature, for example. A record of the intentions helps people who come along later to understand the system, and the particular component’s role in it.

Finally, a specification serves as a sort of contract between a component and the rest of the system in which it functions. The people building the component in question can proceed to work on their component with confidence that the result will likely integrate correctly into the system as long as they build to that contract. The people building other parts of the system can likewise proceed with reasonable confidence that when they go to use the component, it will do what they expect.

33.1.1 Good specification properties

A specification is used for several different tasks by different people over the course of a project. A good specification needs to be structured and contain the information needed to support these people.

Specifications should be clear and unambiguous. Each of the people who will read and use each specification need to come to the intended meaning of the specification.

They should be testable. Someone using the specification should be able to look at a design or implementation and determine whether it is compliant with the specification. That does not mean that determining compliance is easy; it only means possible. Sometimes the most that is possible is to build a body of evidence that a design is highly probably compliant. For a specification to be testable, however, the specification can’t contain statements like “approximately” or “fast” or “heavy”; it needs specific values that define what “approximately” (“+/- 10%”), fast (“at least 20 m/s”), or heavy (“greater than 5 kilograms”) mean so that compliance is not a matter of subjective judgment that can differ between two different people.

The specifications need to be organized. A specification is no good if the people who need to use it don’t know it exists or can’t find it. A specification is also not useful if the people who need it can’t tell whether it is currently applicable, outdated, or a speculative proposal. Specification should be kept in one place where everyone on the project can find all of them, and they should be maintained under configuration management.

A good specification is minimal. It addresses the needs for the system or component that have been identified in the concept work leading up to the specification, but it does not add other elements that are not relevant to the identified needs. (Note, however, that the process of developing a specification can often reveal needs that were missed in building up the concept and CONOPS. When those gaps are found, the concept and CONOPS need to be updated as well as addressing the gap in the specification.)

33.1.2 Specification versus documentation

Specification and documentation play different roles. Specification is a record of what something should be, while documentation is a record of what it has been designed and implemented to actually be. Specification deals with the black-box, external behavior, while documentation deals with the internals of the component. The documentation should connect decisions about the component’s internal structure to the external behavior or structure documented in the specification.

33.1.3 Specification needed to scale a project

A small project, implemented by a very small group of people over a short time and thereafter left alone, and that does not provide safety- or security-critical functions, does not necessarily need specification.

Unless all of those conditions hold, some level of specification is necessary in order to communicate between people and across time.

The communication includes:

• Between those who design the system at large and those who design and implement a specific component. The specification allows different people to work on different parts of the system, which in turn allows a larger team to do the work, and allows some of the work to proceed in parallel.
• Between those who design and implement a component and those who verify its correct behavior. The specification defines the “correct behavior” that verification checks.
• Between the original designer and someone who must adapt part of the system later (for a new feature or to fix a problem)—bearing in mind that those may be the same people, but later in time when the details are likely to have been forgotten. The specification conveys a simplified, abstract version of what a component is supposed to be, so that the designer making changes can work out how the behavior should change and so that they can work out whether the changes will likely integrate correctly to address the new feature.
• Between the system designers and implementers and those who later inspect the system as part of safety or security incident response. Accident response relies on being able to reconstruct both what happened in an incident and what was supposed to happen. This reconstruction is part of determining what changes need to be made to ensure that similar accidents do not recur, and it is also part of working out how to repair things after an incident (when that is possible).

33.2 Specifications and systems

A specification defines the metaphorical shape that the component should have in order to fit into and support the system.

A specification treats the component as a black box: it considers only how the component should be seen from the outside, without determining how the component’s internals should be designed or implemented. One way to look at the specification is that it defines a contract between the system and the component: if the component behaves according to the specification, the system should work correctly as a whole.

A specification may define behaviors or attributes that in effect narrow the range of possible designs, possibly to only a single design. That situation in itself does not make a specification invalid. However, the specification should not include definitions that are not strictly needed to record needed external behaviors solely in order to constrain the design.

After a component has been specified, design of the internals of that component begins. The internal design often uses sub-components. The designers will develop specifications for the sub-components.

This process repeats recursively to lower and lower components, until one reaches components that have no further sub-components. The result is a tree (or possibly a DAG) consisting of alternating layers of specifications and designs. (This has been called the “layer cake model”.) The design of one component (or the system) responds to its specification. The specification for subcomponents depends on the design that has been selected for the component—the design determines both what subcomponents there are, and how they are to work together.

33.3 Example

Some years ago, I worked on a rack-mounted computing system that had high reliability and uptime goals. A decision was taken to include a battery pack in each server assembly, so that if the mains power went out the servers would have enough time to record their state on storage before shutting down.

Consider the specification for the battery pack. It may seem simple—provide enough power to run the server assembly for some period of time—but the actual specification contains several subtle elements because its function is entwined with other system-wide reliability and safety behaviors.

Here are some of the system behaviors that affect the specifications for the battery pack:

• Keep the server assembly running for long enough to save state to storage and perform an orderly shut down
• The server assembly must be able to change its behavior when the battery pack does not have enough energy stored to safely shut down
• A server assembly should function correctly for at least X years before needing replacement
• When a server assembly is developing a condition that suggests a component will fail soon, the server should detect the likely failure and initiate a repair call
• The server assembly must be safe and convenient for a customer to assemble (if needed) and install
• The server assembly must not cause a fire, release toxic gasses, or cause other similar harms (which references a safety standard defining the harms to be avoided)
• The server assembly must not be vulnerable to supply chain attacks, such as substituting unapproved components, in the period between manufacture and installation at a customer site
• The server assembly must fit into a standard 19” computing rack (which references a mechanical standard specification)
• The server assembly is intended to operate in a data center environment (which references an environmental specification)

These are rough objectives for the server assembly as a whole. These translate into specifications on the battery pack itself.

Addressing keeping the server assembly running:

• The battery pack must provide power at X volts and Y current for at least Z minutes when mains power stops providing power to the server assembly
• The voltage and current must be maintained within X of these reference values
• The battery pack must provide connectors to deliver and receive electrical energy to and from other components in the server assembly

Addressing the server assembly changing its behavior:

• A management system attached to the battery pack must be able to determine when the battery pack state of charge is sufficient to meet the voltage, current, and time need, and when the state of charge is not sufficient ⇒ The battery pack must provide information that allows accurate estimation of its state of charge (e.g. having an accurate relation between voltage and state of charge)
• A management system must be able to account for temperature and battery age in making this determination ⇒ The battery must have predictable changes in behavior as temperature and age change

Addressing the server assembly lifetime:

• The battery pack must not degrade its storage capacity more than X% over Y years
• The battery pack must support a management system detecting when the storage capacity degradation is approaching a level when battery pack replacement will be needed

Addressing likely failure:

• The battery pack must exhibit behaviors that allow a management system to detect degradation or incipient faults, such as failure of one battery cell or overheating
• The battery pack must have identification that allows batteries from the same manufacturer and manufacturing lot to be tracked for common failure patterns

Addressing safe and convenient customer installation:

• The battery pack should be shipped discharged
• The battery pack should be safe to handle without protective equipment as long as its case or assembly is not damaged or modified
• The battery pack should not make the server assembly too heavy for one person to lift safely
• The battery pack, if separately installable, must not have error-prone cabling that a customer might hook up the wrong way

Addressing fire, toxic gasses, and similar safety issues:

• The battery pack should be sealed against emissions up to at least X internal pressure
• The battery pack should be able to handle up to X impact without taking damage that would lead to thermal or mechanical failures
• The battery pack should provide sufficient information to a management system that the management system can disconnect the battery when a failure is beginning, and so that the management system can trigger external safety systems

Addressing supply chain attacks:

• The battery pack must be handled only by trusted employees and agents of the company from manufacture to installation
• The battery pack must include identifying information that is hard to falsify or replicate

Addressing fitting into a standard rack:

• The battery pack must weigh no more than X kilograms
• The battery pack must occupy a volume no greater than X by Y by Z millimeters
• The battery pack must connect to the rest of the server assembly using X brackets

Addressing the environmental conditions:

• The battery pack must function nominally in a given temperature, air pressure, humidity, air gas composition, and dust environment

These example objectives are not all of what would be needed for a server battery pack, but they illustrate several of the kinds of concerns that the battery pack’s designers will need to consider. These rough objectives must be turned into more precise specifications in order to guide the designers accurately. For example, some of the statements above use subjective words like “nominally” that need to be made precise. Other statements are too general and need to be decomposed into a set of more specific statements.

33.4 Kinds of specifications

“Specification” is a deliberately broad term, encompassing many different ways of recording what something should be or do (and why).

Many people assume that “specification” means “requirements”. While requirements are one kind of specification, they are not the only one—and requirements are not generally sufficient by themselves to record all the information needed about behavior or structure.

Kinds of specification include:

• Scope or environment. This is a specification of the boundary for a component (or system) Section 6.3 and, therefore, what is inside and outside of the component (or system). The nature of the environment can be seen as a statement about interfaces, it is usually a good idea to document this explicitly. XXX say more about scope below
• Requirements. These are short prose statements about what is expected of the component. These statements have the advantage that they have all the flexibility of written language. A reader only needs to know normal language and the associated technical jargon to understand the meaning of the statement. Requirements also have the advantage that they are easily organized and identified in an outline or table form, so that one can trace through each individual statement. Requirements inherit the disadvantages of written language that lead people to devise other notations or diagrams.
• Interfaces. While a specification is in general a definition of the overall interface between a component and the system around it, in many domains part of the specification is gathered into an interface (often recorded in an Interface Control Document). A software application programming interface, which defines data structures, function calls, and communication steps used to communicate with a component, is one example. An environmental specification is another example that specifies the heat, vibration, atmosphere, electromagnetic radiation, dust, and other environmental conditions that a component must handle correctly.
• Models. These are (semi)formal expressions about structure or behavior, written in a specialized notation or diagram. These expressions have the advantage of being able to express needs that are not easily expressed in prose, such as physical or temporal relations, or mathematical statements. These have the disadvantage that readers often have to learn new notations in order to understand what the expressions mean. Many notations do not lend themselves to easy enumeration of each individual condition that the component should comply with.

There are many kinds of models.

• Mechanical models show the physical interface that the component should support. This can be as simple as a volume in which the component must fit. It might include connectors for how the component is attached to other components in the system. More advanced models might include the forces that may act on the component, and how the component must pass those forces from one place to another, or how a component may deform in response to forces.
• Mathematical models document how a component behavior should respond to outside inputs, expressed as mathematical formulas. This can range from a simple linear response to an input to a complex and stateful impulse-response model. Many of these are expressed on a continuous time domain.

• Behavioral models document how the component will perform sequences of output actions in response to inputs. Many of these models are expressed as diagrams: state machines, sequence or activity diagrams, or timelines. Some models, such as state machines, express all the possible sequences of behavior that could happen; other models, such as sequence diagrams, express one or a few specific sequences without being exhaustive. Many of these models operate on a discrete time domain.

• Domain-specific models. These express needs that are specific to some problem domain, and that are not easy to express in one of the other kinds of models. For example, Earth-orbiting spacecraft typically have an orbit specification, which can be used to determine where the spacecraft will be at various times in the future and what the conditions will be at those times. A material specification might include parts of the phase diagram for the properties of the material at different temperatures and pressures, indicating some conditions in which the material must be solid or gas. Most domain-specific models have an underlying mathematical definition, but the needs can be expressed more clearly using the domain-specific notation rather than only using the underlying formulas.

33.4.1 Combining multiple kinds of specification

In practice I have found that no one kind of specification meets all needs, and have used multiple kinds of specification together.

Generally, each kind of specification we use meets the good specification objectives of being clear and testable, as defined earlier.

Mixing multiple kinds of specification, however, requires care in organizing the specifications. Different kinds are often written and stored in different tools (a tabular tool for requirements; a CAD tool for mechanical drawings). This easily leads to a situation where a practitioner cannot find all of the specifications to which they need to be paying attention.

One way we have addressed this is to use a table of textual requirement statements as a primary specification, and include requirements like “the component shall comply with state machine X”, including a reference to the drawing of the state machine. Using a tool that makes all these forms accessible through one common user interface helps make this convenient for users. Using tools that can perform configuration management across all the different forms of specification also helps.

33.5 Using specifications in a system

We first look at how specifications are developed and used from the outside: from the perspective of those who are concerned with how a component fits into the system, and not with what the specification means for the design internal to a component.

A specification for a system derives from the objectives and CONOPS developed during the system concept development phase.

The system-level specification leads, in turn, to a system design and then recursively to the concepts and specifications for components in the system.

33.5.1 Building the specification

This is the first step in using specifications. The specification developer looks through all of the conceptual material assembled for the system or for a component, and organizes and formalizes it to make a specification.

In practice this does not happen all at once. People develop the various kinds of objectives that lead to the specification iteratively, and parts of the specification will be developed as the objectives and concept becomes clear. As people develop the specification, they will identify gaps in the concept, which will lead to improvements in the objectives and CONOPS and in turn lead to updates to the specification.

33.5.2 Evolving the specification

The needs that a system solves change over time. New capabilities get requested. Regulations evolve. Problems with the system are found and need to be fixed. All of these can lead to changes in the concept and thus to changes in the system specification.

The concept and design of components also changes, and for similar reasons. As well, a component may have a perfectly adequate design, but it may become outdated because subcomponents become unavailable. This leads to a redesign of a component, inducing new specifications for subcomponents.

It is important to follow an organized process when a specification changes. Many process standards recommend specific approaches; for example, ISO 26262 [ISO26262] specifies that any change to a system must begin with an impact analysis, which determines how a change to objectives or specification propagates through the design of the system, and downward through the hierarchy of components. Standards like that also specify that the specifications and designs be maintained under configuration control so that everyone can know whether a change is a work-in-progress proposal or has been committed to.

33.5.3 Validating the specification

The specification must reflect all of the needs identified in the concept from which it derives, and the specification must not add needs that do not appear in the concept and objectives. Before a specification can be declared complete, someone must go through all the material in the concept to check that the specification accurately reflects each of the identified needs or objectives.

A specification validation exercise can also help identify gaps in the objectives. Checking the specification often involves someone who was not part of developing the objectives and CONOPS; a fresh perspective can lead to asking questions about the objectives or the specifications that in turn lead to discoveries of topics that are missing.

33.5.4 System consistency

As the system design grows and more and more components are defined and specified, someone needs to check that the designs and specifications are all consistent. This is especially important for “long distance” dependencies: where the correct function of one component depends on the correct function of another component in a different part of the system. (More formally, when two components A and B depend on each other for correct function, and the lowest common parent of A and B in the component hierarchy is near the top of the hierarchy.)

33.5.5 Safety and security design

As we will discuss in future chapters ! Unknown link ref, the safety and security properties of a system must be designed top down, and they need to be defined early in system development, before too many low-level components are designed.

We advocate using the systems safety methodology ! Unknown link ref, which emphasizes starting with the accidents or losses that are to be avoided, and then the conditions that must be maintained in a system to achieve safe operation. (This is different from many safety methodologies, such as functional safety, which focus on safety in the face of failure conditions and do not address safety problems arising from design or component interactions.) The categories of losses come from the safety and security objectives defined in the concept development phase.

Some example conditions:

• A spacecraft must not perform any propulsive maneuvers, emit radio frequency radiation, or perform any physical deployments within 1 km of the upper stage vehicle from which it was deployed. (This example is based on real launch vehicle safety requirements.)
• A planetary lander must maintain engine thrust until it has touched down on the surface. (This example is based on the accident analysis of the Mars Polar Lander spacecraft; see ! Unknown link ref for details.)
• An autonomous aircraft must not take operational commands from an unauthorized source.

Once these conditions are identified, systems engineers must determine how to address them in the design of the top-level system. They must then create derived specifications for each of the top-level components in the system, and show that if each of the components meets its specifications the overall system will exhibit safe or secure behavior by complying with the safety and security conditions. This process is repeated through at increasingly lower levels of the system.

33.5.6 Review and approval

A specification guides the design and implementation of parts of the system. Given the importance of this role, a specification—or an update to a specification—should be reviewed before being committed to. Each specification should be checked by the people whose work it affects: system designers, the designer of the component or system that contains the thing being specified, potential implementers, and those people who are working on components that will interface with or use the component being specified.

As with other system artifacts, a specification or specification update should be under configuration management so that each user can determine whether they are using the correct version or not, and whether the version they are using is a proposed or work in progress version, is the current approved (baselined) version, or a version that has become obsolete.

33.6 Using specifications for a component

We now turn our attention to those people and activities who use a component to design and implement a component; that is, who are concerned with how the internals of a component reflect its specification.

There are two tracks of activity that use a component’s specification:

One track follows the design and implementation of the component itself, which should result in a component that complies with the specification. The other track follows the design and implementation of verification methods, such as tests or static analyses. The tracks come together when the implementation gets checked by the various verification methods, resulting in a determination of whether the implementation is in fact compliant, or whether the design and implementation need to be fixed to bring it to compliance.

33.6.1 Learning about a component

A specification is an abstracted view of what a component should be. That makes it useful as a guide for someone who needs to learn about a component, before diving into the design or implementation of that component.

Someone who is learning about a component—or about the structure of the system across many components—needs to be able to find the relevant specifications. The specifications should be organized to support them:

• Components should have names or identifiers
• Specifications should be stored in a repository that allows people to find a specification by its component id, or be able to navigate through the structure of the system to discover information about components
• Users should be able to start their search in a single place, well-known to all the team, without having to discover multiple tools that might or might not be readily apparent to new team members

33.6.2 Designing and implementing to specification

The general task of a designer or implementer is to create a component that complies with its specification. In practice, of course, this is a complex activity.

The designer needs to be able to clearly identify all of the behaviors or capabilities that the component must implement. This implies that the specification must be organized in a way that helps the designer find all of these, and in a way that can serve as a checklist for tracking which features have been satisfied and which have not yet.

As we will discuss further in upcoming chapters, the designer or implementer should be able to identify which aspects of the component have the highest design risk or are the most technically complex. The designer and implementer will often choose to focus on these hard aspects first, before dealing with aspects that are easy to solve. The hard aspects are often candidates for prototyping, in order to determine if a design approach is feasible and can meet the specification. (See XXX for more on prototyping and risk reduction.)

Complex systems and components can benefit from the combination of incremental development and continuous integration. Incremental development involves selecting a few parts of the component’s specification and implementing those, followed by testing. Once those aspects of the component appear sound, the developers perform a second iteration by selecting a few more aspects of the specification and adding them to the design and implementation. Continuous integration, in this context, involves performing integration testing of these partial designs and implementations in a skeleton of the rest of the system. The partial implementation of this component may use mockups of subcomponents, or interact with mockups of peer components in the system. We discuss incremental development and continuous integration more in XXX.

As people work through design and implementation, they are likely to find problems or gaps with the specification. The specification may be ambiguous in some part, or the specification may not define the behavior for some condition. The developers must be able to work with those who defined the specification to sort out these issues. The developers should check the specifications in depth, asking the specifiers questions to check their understanding or to confirm that there are issues. The developers then should work with the specifier to resolve the issues.

The developers should not make an assumption about a gap or ambiguity and move forward without confirming their assumption. The people who wrote the specification are responsible for ensuring that the specifications for different components are consistent and address large-scale safety or security concerns. The behaviors needed to support correct interaction are encoded in the specification. The developers are responsible for implementing components that correctly support these behaviors so that the resulting system works correctly. The developers do not necessarily have the big-picture perspective to make changes to these critical behaviors, and do not necessarily know who else needs to know about an assumption in how a component is defined. The developers need to work collaboratively with those responsible for the specifications so that all the pieces of the system remain consistent and correct, and so that everyone involved shares a common understanding of how the components and system are to function.

A component’s implementation will need to be verified against the component’s specification. People using continuous testing or test-driven development methods have had good results producing correct component implementations efficiently by testing an implementation in small increments as functionality gets added to it. This reduces the risk that the design or implementation has made some fundamental, early mistake that becomes increasingly expensive to correct as more functionality is implemented on top of the erroneous implementation. Performing continuous testing (or verification) requires having verification cases defined and implemented concurrent with the implementation of the corresponding functionality.

Finally, each component design and implementation will need to be reviewed and approved before being accepted as finished. Verifying that the design and implementation comply with the specification is a major part of the review process. The review activities will be much easier if the specification is well organized.

33.6.3 Evolving specifications

As mentioned earlier, a component’s specification will likely change when a system remains in use for a long time. Systems engineers will need to investigate the impact of making a change to a specification before committing to the change.

The component designers and implementers are part of the investigation process. While a systems engineer can look at what will change in how a component interacts with other parts of the system, the component designers and implementers are better positioned to evaluate the effect that a change in specification will have on implementation or verification.

To change a design and implementation in response to a change in specification, the developers need to correctly determine what has changed in the specification. Having a clear mechanism for showing what requirements have been removed, added, or changed, and for showing specifically how other parts of the specification have changed, makes this task possible. In particular, being able to accurately enumerate every change is important; the developer should not have to hunt for subtle changes that may be hidden.

The decisions that are encoded in a component’s design include how different parts of the component interact with and depend on each other. When a component’s design is to be changed in response to a change in specification, some parts of the design will be directly affected. For example, a decision to add a new input message to a component directly implies that new message reception and handling functions must be implemented. However, one change can affect other parts of the existing design, and the designer and implementer must find and address all of these effects. The example new input message, for example, might require changes to a database schema for storing additional information, or might affect response time behaviors that require changes to foundational concurrency control capabilities in the design. Having a clear record of how parts within one component are designed to depend on or affect each other reduces the effort involved in making this kind of change, and reduces the chances of an error stemming from some dependency being overlooked.

33.6.4 Verifying a component

The specification defines what a component should be or do; the design and implementation define how it is or does these things. Verification is the process of ensuring that the implementation produces behaviors that match the specification.

Every element of the specification should have a corresponding method for verifying compliance of the implementation. Different aspects of the specification will require different methods: some aspects can be verified by testing, such as showing that given some input A, the component responds with behavior B. Other aspects will require demonstration, such as showing that a physically representative user can see and reach control devices. Some aspects—especially safety and security—can only be verified by analysis or formal methods, such as showing that a component never enters performs some action identified as unsafe.

Verification methods involve design and implementation, similar to the design and implementation of the component itself.

Designing a verification method involves, first, determining how a specification property can be verified. (Sometimes a property is best verified using more than one approach in parallel.) once the approach—testing, review, demonstration, or analysis—has been determined, the next step is to design how that specific specification property will be checked. That can involve designing a set of test cases that cover the expected behaviors, or defining a test procedure to evaluate a mechanical component, or defining who will perform a review and what they will look for.

Implementing a verification method turns the design into a specific set of tools and actions that, when used, give a yes-or-no answer to whether the component is compliant.

The verification methods can have errors. Indeed, in some cases the verification of a property can be more complex than the component implementation it is checking. This means that the verification designs and implementation need careful scrutiny to ensure that they are, in fact, checking the specified properties and not something else.

The verification methods also must be complete: if some property is worth specifying, it is worth verifying. The verification designs and implementations need to be checked to ensure that they cover all of the specification. Explicitly recording which parts of the specification any particular verification method checks helps the task of checking completeness.

Finally, it is common for project management to track what portion of a component’s specification has been completed and verified. This can be organized by identifying each property in the specification, and tracking which verification methods check each one. As verifications are done, the project managers can determine which parts of the specification correspond to verification activities that passed.

33.7 Specification artifacts

Specification activities take as input the objectives ! Unknown link ref and CONOPS ! Unknown link ref artifacts that were generated during concept development.

The specifications themselves involve:

• The component breakdown structure ! Unknown link ref, which organizes the components that make up the system and define the entities that will have specifications written for them
• For each component, some kind of index or table of contents that organize all the elements of the specification for that component
• Tables of requirements ! Unknown link ref, interface definitions ! Unknown link ref, and models ! Unknown link ref that make up the specification

The elements in the specification should include traces that show how each individual part of the specification derives from some part of the objectives or CONOPS, and conversely how each part of the objectives is reflected in the specification.

The specification artifacts should be maintained under configuration management. That means that there should be a common repository that everyone working on the system can use to retrieve (and potentially update) the artifacts. The repository should maintain separate versions of each artifact, and clearly identify which version is the current, baselined version that people should use, which versions are outdated, and which are works in progress.

The configuration management system should support people reviewing a specification, and must support recording when a particular version has been approved to be baselined.

Chapter 34: Requirements

34.1 What are requirements?

Requirements are one kind of specification: they say something about a property that a component or system should have, or a behavior they should exhibit.

A requirement is a specification in the form of a single, declarative textual statement. In the simplest case, a requirement is a statements of the form:

For example,

There are many nuances and variations on this basic form, but they are all extensions of this basic idea.

Requirements are written this way in order to maximize the simplicity and clarity of the specification.

Requirements are only one part of the specification for a component or system. They document specific facts about a system’s design, but they do not document the explanation of how that particular design came to be. They do not document the general purpose and scope of a particular component. They do not document complex interaction patterns. These other parts of a specification are documented in other design artifacts that complement requirements.

34.1.1 Why write requirements?

One of the jobs of systems engineering is to ensure that a user or consumer of some artifact (system or component) will be satisfied with the artifact once it is built and deployed.

The specifications for a system or component serve as a way to organize the information about what the user wants, and to organize the process of checking that the final result meets the user’s desires. The specification thus acts as a kind of implicit contract between the end user and the implementers: if the user agrees that the specification properly records their objectives, and the resulting system can be verified to meet the specification, then then the implementers have built something that satisfies what the user agreed to. (Whether the user is actually satisfied is a separate matter.)

XXX would a couple diagrams help here? A first one might show user → conceptual artifact, conceptual artifact → developer → concrete artifact; a second one might show systems and verification in the picture?

This means that there are three main uses for requirements (and the rest of specifications):

1. Encoding the user’s objectives in a written form, and allowing the user to validate that the specification matches what they want.
2. Guiding implementers as they work out the design for the artifact that will meet the user’s objectives.
3. Providing a checklist for verifying that the resulting artifact meets the specification, and thereby the user’s objectives.

A systems engineer is typically the keeper of the specifications, responsible for overseeing the writing, changing, and verification of requirements and other specifications.

Requirements—and all specifications—are therefore acts of communication between multiple groups of people with different roles in building the system.

Systems engineers are facilitators and interpreters in this communication between users and implementers. They are responsible for translating information received from users into specifications (including requirements), for explaining the specifications back to the users for validation. The information from the user is often unstructured and incomplete. It is up to the systems engineer to work with the user to clarify their objectives and ensure that the result accurately reflects the user’s intent. The systems engineer also works to ensure that the specifications are complete. This often involves identifying use cases that the user has not thought of themselves and working with the user to define what behavior the system should have in those other cases.

The systems engineer also facilitates the implementer’s work. The systems engineer develops specifications so that the implementer has a clear guide to what they need to design and build; this requires that the systems engineer provide translation or explanation when the specification does not use the same terms or concepts that the implementers do. The systems engineer is also responsible for ensuring that the final artifact meets the customer’s objectives by overseeing the verification of the implementation against requirements (and other specifications). This involves working with verifiers to ensure that verification methods match the requirements, and checking that all requirements have been verified before the system is declared done.

A systems engineer performs other tasks using requirements, such as checking consistency or completeness. We will discuss these tasks in a later section.

A good requirement must meet several objectives in order to provide accurate communication between all these parties:

• A requirement must be unambiguous. Each of the parties must be able to read the text of a requirement and have the same understanding of what it means.
• A requirement must have an unambiguous meaning about the system.Two people should be able to look at the system and read the requirement, and come to the same conclusion about how the system is supposed to behave if the system meets the requirement. Many people express this as “a requirement must be verifiable”.
• A requirement must be accurate. A requirement that misleads a reader has no point.

These needs lead to conventions about how requirements are written and organized, as we will discuss later.

34.1.2 What are requirements about?

Requirements are a general-purpose way of writing down facts about what something is supposed to be (or not be).

Requirements can apply to just about anything. In a typical system project, they will be used to:

• Record the objectives that end users have for the system as a whole
• Record the regulations and policies that people governing the system
• (or its market) impose on systems like this
• Record the behaviors and properties that individual components need
• to have on order to fit together into the system as a whole
• Record the processes that the system’s implementers must follow
• while building the system
• Record elements of the contractual relationships between the
• system’s implementer and vendors that supply parts or services
• Record policies for the behaviors required of people working with the system

34.1.3 The context for requirements

Requirements don’t stand on their own.

Most requirements in a system will apply to particular components in the system. The component breakdown structure provides the list of components that requirements can be about.

Requirements are part of more general specifications for the system and its components. The specifications include

• A definition of the scope and purpose of each component
• The structure of how components connect together to make up the system
• Analyses of the system design, such as fault trees or performance models
• Specifications of behaviors that cross many different components

The requirements must be consistent with these other parts of a component’s specification.

In the end, requirements are satisfied by the implementation of the components in the system. Being able to trace the connection from a component’s requirements to the pieces of the implementation matters in order to be able to show that the requirements are satisfied.

34.2 A single requirement

A requirement itself is a single statement about something that should be true about something.

More formally, a requirement has three parts:

• A subject, naming what thing the requirement applies to. The subject can be the whole system, a part of the system, or sets of parts.
• A mode, naming the relationship between the subject and the properties. A word like “shall” or “must” is the most common mode, indicating that the property must be true or shall be true about the subject.
• A property, encoding a predicate that is to be true about the subject. This can be just about anything, as long as everyone involved can agree on what it means for the property to be true. It can describe a behavior that the subject should provide, or some characteristic that the subject should maintain. The property is usually written as a verb followed by the object of the property, such as

where “be placed” is the verb.

Some examples:

• The spacecraft shall have a mass no more than 1 kilogram
  • Subject: the spacecraft as a whole. The spacecraft is made up of many subcomponents; this requirement applies to all of them put together into the complete spacecraft system.
  • Mode: shall—this is something that we require to be true.
  • Property: have a mass no more than 1 kg. This is a measurable property of the spacecraft; once assembled, it can be put on a scale and checked.
• The electrical power subsystem (EPS) must provide a minimum of 25 W in standby mode
  • Subject: the electrical power subsystem. This is a named subsystem within a spacecraft.
  • Mode: must—this is something that we require to be true.
  • Property: provide a minimum wattage when in a particular mode. This is a conditional property: it only applies when the EPS is in standby mode. For this property to be meaningful, the design of the EPS must include a set of modes, and “standby” must be one of them.

34.2.1 Example

Consider an example of a statement of what the mission manager for a small spacecraft mission wants:

This sentence has a number of problems. It mixes statements together: the mission and the spacecraft, the operating environment and the lifetime. The sentence is not very precise: what is “low Earth orbit”? What does the spacecraft have to do to “operate”? It is unachievable: nobody can guarantee that a spacecraft will function for a particular duration as an absolute guarantee; what if there is an unusual solar flare that fries its electronics?

We can improve the example sentence a bit by splitting it into three requirements statements:

• The mission shall operate a small spacecraft
• The spacecraft shall operate nominally for at least three years with 95% probability
• The spacecraft shall operate nominally in low Earth orbit between 200 and 400 km altitude

These requirements improve the original statement. First, it splits the original so that each requirement is about a single topic (and is written in the subject-mode-property form). Second, it improves the description of two of the requirements by making them more achievable (“95% probability”) and precise (altitude range given).

These three requirements in themselves are not sufficient. Before the requirements are done being written, for example, there will need to be a definition of what “operate nominally” means. Similarly, the “at least three years” requirement is not enough by itself: three years would be difficult or impossible to meet if the intended environment were the surface of Venus; it would be almost trivially easy in the intended environment were an air conditioned clean room. Adding more information about the environment is necessary to interpret the three-year condition—for example, what is the expected radiation environment at those altitudes?

The three example requirements are not sufficient in another way: they are high-level and provide the designer of, say, a battery subsystem no guidance about how the battery must be designed so that the spacecraft meets these requirements. The derivation or flow down is the topic of an upcoming section.

34.2.2 Rationale

A well-written requirement is concise. As such, it makes a statement about what a component should do—but the text of the requirement does not capture why the component should do that.

Good requirements should include a rationale statement that documents the thinking that went into choosing to make the requirement. The rationale does not change the requirement; it only adds explanation. The rationale helps those who must come along later, after the requirements are written, to understand or evaluate the requirements. It helps educate other engineers about considerations that may not be obvious. It helps those who later need to revise requirements understand what constraints there may be on the requirement they are changing.

34.3 Multiple requirements

Requirements actually come in groups; they are practically never singular.

The meaning of a group of requirements is the logical and of all of them. If there are ten requirements, an implementation complies with the requirements if it complies with all ten of them individually.

There are two issues to watch out for when there are multiple requirements: contradictions and exclusivity.

34.4 Organizing requirements

Even a moderately-sized system will typically have thousands of requirements. Users need some kind of organization of all those requirements in order to find the requirements they will be working with.

There are three concepts to discuss: organizing by subject, organizing by sections, and hierarchical writing.

34.4.1 Levels of requirements

People use requirements for different purposes. This leads to fundamentally different kinds or requirements.

At the most abstract level, the general product or mission objectives capture what stakeholders want the system to do—its purpose. These almost always start as general, vague statements. The stakeholders, system engineers, and product managers refine these over time into a clearer definition of the system’s purpose. The exercise may or may not result in proper requirements statements, but it is worth treating the results as if they are requirements and showing how the top-level system requirements derive from these objectives.

Projects also have guiding objectives that do not specify the system directly, but instead define policy or standards that the system must adhere to. There are many kinds of policies, including:

• Business policy: common rules for how the organization operates; expected profitability margins; expected customer base
• Security policy: what kinds of hazards, threats, and attacks should be addressed in the system design; what should be protected; rules for prioritizing different attacks
• Safety policy: what entities should be considered for safety; what hazards to consider; acceptable incident rates for different hazards; can include things like launch assurance standards
• Regulatory policy: regulations and laws that the system must comply with
• Development standards: standards that govern how the team designing and building the system should go about their work, including quality standards such as ISO 9001 or process standards such as NASA NPR 7150.2
• Other standards: industry or government standards that will inform the design of the system, such as standards for hardware components or for human-machine interface design

It is helpful to organize the product/mission objectives and all the various policies and standards into separate collections, identified by the kind of policy or source of objectives. For example, one can maintain one collection for business policy and a separate one for the quality assurance standard being used to build a system.

The top-level requirements on the system as a whole are part of the formal or semi-formal definition of what the system is to do. These requirements say what the system is and does when looked at from the outside, as a black box. These requirements are best kept separate from the more vague product/mission objectives—the objectives represent desires, while the top-level requirements represent the commitments made for what the system will do. The derivation mapping from objectives to top-level requirements provides a place to record the rationale for why different decisions were made about the commitments in the system, and why the decision was made not to commit to supporting some desires, represented in objectives.

Requirements on lower-level components provide definitions of what the pieces that make up the system must do. These obviously have a different scope than the top-level requirements for the whole system.

34.4.2 Organizing by subject

The first concept is that requirements should be organized by their subject, following the component breakdown structure.

The system objectives are those requirements that apply to the system as a whole. These typically encode the CONOPS for the system, along with requirements derived from the process or design standards.

The rest of the requirements apply to specific components within the system. The component breakdown structure defines what the components are, and gives them names.

Organizing by component is important for proper verification, so that each requirement can be connected to the implementation artifacts that are expected to comply with the requirement, and so that the implementer of some component can properly determine all the requirements they need to adhere to.

34.4.3 Organizing by section

One single component or process/design standard can often have several hundred requirements. Users can find and work with all these requirements more easily if they are organized by topic as well as by subject.

This can be done by creating a set of topic sections within each component. Often these sections are the same for all components—sometimes empty when they are not relevant, but having the same organization across all components help people find what they are looking for.

There is no one recommended set of sections that will apply to every system. The choice of sections is affected by the kind of system or components being developed, as well as by process and design standards. For example, if an automotive project is following the ISO 26262 Functional Safety standards [ISO26262], the Safety Goals and/or Safety Requirements should be collected into one section.

As a starting point, we have used variations on the following set of sections in several projects:

• Operational requirements. These relate to how the component turns on and off, and the major operational modes of the component.
• Functional requirements. These record the behaviors or functions that the component is expected to provide (when it is turned on, if that applies)
• Service or life cycle requirements. These address how the component is to be manufactured, installed, maintained, and retired
• Budget limitations. This includes data rates, mass, power consumption, RF emitted or received, heat emitted, or gas/vapor emissions
• Environment. What the component must be or do in its environment. For example, vibration resistance, gas or dust resistance.
• Discipline-specific sections. This includes sections for safety, security, and regulatory compliance.
• A miscellaneous section for requirements that don’t fit into other categories.

It’s a good idea to work out one or a few section structures that work for your project, then use those sections consistently across all components.

Keep in mind that some requirements will always fit into multiple sections. For example, a requirement may both be about regulatory compliance and define a function the component is supposed to provide. Try to make consistent choices about which section a requirement goes in, but don’t try to make some perfect hierarchical section scheme that would let people avoid making such choices.

34.4.4 Hierarchical versus flat requirements

There are two general structures for organizing requirements on a particular topic:

• As a single-level or flat collection of requirements, all at the same level
• In a hierarchical or outline structure There are pros and cons of each approach. The choice of which you follow will depend on the style of working in your project, the capabilities of the tools you use, and the process requirements that apply to your organization.

The flat organization has all requirements within a section be at the same level. Each requirement is independent of the others and can be understood only by reading the text of the requirement.

The hierarchical organization places requirements into an outline, with general requirements and more specific sub-requirements. The sub-requirements must be read and understood in the context of their parent. The sub-requirements provide details, clarification, or limitations on the general parent.

Consider a set of requirements for security on a TCP/IP communication channel. The general requirement is that the communication channel should be authenticated and encrypted. In outline form, this looks like:

1. Communication channel X must implement security mechanisms
   1. Communication channel X must require authentication before application data is exchanged
      1. The authentication protocol must mutually authenticate both parties to each other
      2. The identity being authenticated must be granted by the organization’s security management system
      3. The authentication protocol must be resistant to man-in-the-middle attacks
      4. The authentication protocol must support revocation of either party’s credentials within X minutes
   2. Communication channel X must maintain integrity and confidentiality of the application data being exchanged
      1. The confidentiality protection must be resistant to traffic analysis

Consider requirement 1.1.1, requiring mutual authentication for the communication channel in question. The requirement for mutual authentication must be understood only to apply to communication channel X. There could well be another communication channel, called Y, that does not have the same authentication requirements.

Written in a flat style, the requirements might be expressed as:

1. Communication channel X must require authentication before application data is exchanged
2. The authentication protocol used for communication channel X must mutually authenticate both parties to each other
3. The authentication protocol used for communication channel X must use identities granted by the organization’s security management system
4. The authentication protocol used for communication channel X must be resistant to man-in-the-middle attacks
5. The authentication protocol used for communication channel X must support revocation of either party’s credentials within X minutes
6. The communication protocol used for communication channel X must maintain confidentiality and integrity of the application data being exchanged
7. The communication protocol used for communication channel X must provide confidentiality protection that is resistant to traffic analysis

Each of these statements can be read on their own; each statement includes all the necessary qualifications (“the protocol for communication channel X must…”) to identify the scope of its subject without having to refer to other statements.

There are pros and cons of each approach.

• (Pro) Flat organization allows each requirement statement to be read and understood on its own, without reference to other requirements
• (Con) Flat organization requires some requirement statements to include a lot of redundant text to identify the scope of the subject. The redundancy makes it more tedious to write the statements, and leads to errors where one of the qualifying clauses in the subject doesn’t get edited and becomes inconsistent with the others
• (Pro) Hierarchical organization makes it easier for the reader to scan for a particular requirement, and makes the individual statements easier to understand
• (Con) Hierarchical organization means that sub-requirements do not stand alone, but must be read in the context of their parent.

34.4.5 Requirement identifiers

Every requirement needs a unique identifier.

People use this identifier to refer to the requirement, including using it as a bookmark or link to reference the requirement in other documents. Software check ins to a repository often use the requirement identifier to indicate what functionality is being added to the repository. Task management systems use requirement identifiers to track the progress on implementing and verifying particular requirements. In general, the requirement identifier enables the integration of requirements management with other tools and tasks

The identifier must be stable. That is, once a requirement has been given an identifier, that identifier should not change. The text of the requirement can (and will) change, but the identifier remains a stable way to refer to the requirement in documents, email, and other messages without having to track down all the uses of the identifier and change them.

It is good practice for the identifier to convey some information about the requirement. At minimum, the identifier should make it clear what component or body of external requirements the identifier applies to. If one writes requirements hierarchically, then using the number of the requirement in the outline is a good identifier.

Having the identifier carry some information helps the user check that they are referencing the requirement they intended to reference. It also helps the reader to know generally what the writer is talking about, without going into a requirements management system to check.

For many projects, I have used the format <component id>:<hierarchical requirement number> as the identifier. For example, space.eps.panels:3.4.2 for a requirement applying to a spacecraft’s solar panels.

There are requirements management systems that use a universal, flat namespace for identifiers, such as REQ-82763. This is not a good identifier, because it makes it hard to check when one has accidentally mistyped or miscopied the identifier into another document. If one accidentally types REQ-82764 into another document, that other requirement could apply to a completely different component—and the mistake is obscured.

34.5 Writing good requirements

Requirements are a way of communicating between people on a project: between the customer and systems engineers, between those who look at how multiple systems work together and those who implement the pieces, between those who design and those who test. A good requirement is one understood equally well by all the people who use that requirement.

Writing good requirements takes practice, but the following guidelines will help in writing and reading requirements.

34.5.1 General form

Individual requirements have a general form:

The subject is often a component named in the component breakdown structure. It should be named explicitly:

The majority of requirements use either the word “shall” or “must”, depending on the organization and industry. “Shall” indicates an assertion that the statement about the subject is to be true in the implemented system. “Must” expresses the obligation that the statement will be true in the system. In practice the two words mean the same thing when writing requirements.

The property is a predicate that should be verifiably true about the subject.

Writing the predicate is usually the complex part of writing a requirement. In some cases the predicate is simple:

In other cases, the predicate must have conditions added, saying when or under what conditions the predicate applies:

Sometimes the requirement statement is easier to read if the condition clauses are presented in a different, natural order. However, the semantics remain the same: the clause is part of the property statement:

34.5.2 Single topic

A requirement should specify a single property of the subject. The examples above all deal with a single property.

There are requirements that may have multiple things in their property statement that still deal only with a single property. For example:

Formally, this requirement deals with a single property: what color the widget may be painted. The color is restricted to a set of three colors—but the property in question is the color.

Note that this requirement is slightly ambiguous: it is not clear whether the widget can be painted only one of those colors, or some mixture of them. This requirement could be improved by either rewriting it as:

Or adding a second requirement:

34.5.3 Clarity about subject

A good requirement must be clear about what thing it applies to. In general it is best to write down a proper name of the subject—the name of the relevant component in the breakdown structure, for example.

This rule makes for a lot of repetition in requirements. “The control system must X”, “The control system must Y”, “The control system must Z”, and so on. While it means a little more typing, using the component’s name in each requirement means that each requirement can be understood on its own.

34.5.4 Consistent language

Use consistent terms throughout requirements. Always call component X by one name; don’t change it from requirement to requirement. Always call some one function by the same name, so that it’s clear that all the relevant requirements really are talking about the same thing.

Having lists of names or terms helps those who write requirements to use consistent terms, and provides those who read requirements with definitions when they need to confirm what a term refers to. This means:

• Define the component breakdown structure, and use the component names in that structure when writing requirements
• Provide a list that defines the functions that a component must provide. Use the function names consistently when writing requirements, and provide a definition of the function that will help a requirement reader understand the requirement statement
• Provide a glossary of terms with technical meanings. Use terms consistently with the glossary, and include definitions of terms that are clear enough to explain the terms to readers

34.5.5 Plain language

Requirements (and the rest of specifications) may be written by one or a few people, but they will be read by many people. The readers need to understand correctly what the requirements mean. Many of those readers will be learning about the system by reading requirements or other documents, so they won’t enter into reading the requirements with the same context that system engineers writing the requirements will have.

This means: don’t get fancy with requirements language. There are some ways that requirements will sound stilted, like the subject-mode-property form. There is some technical jargon that is needed to make the requirement precise. But don’t make the language more complex than it needs to be.

For any words or phrases that do not have a meaning that will be obvious to all your readers, help them out by defining how those words are being used in the specifications. Start with “must” versus “shall” and any other mode words (see Advanced Requirements below). Provide a glossary of the definitions of the rest of the words.

34.5.6 Negative requirements and “only”

Many organizations prohibit requirements that say “shall not”. Negative requirements have their place, but they are tricky to get right. The problems arise with exactly how broad or narrow the requirement actually is.

Consider a component implementation that could do one of three behaviors, A, B, or C.

If the component has a requirement “the component shall do A”, the implementation satisfies the requirement (it does A). That is because the requirement, as written, allows for the implementation to do other behaviors as well.

If the component has a requirement “the component shall only do A”, then the implementation does not satisfy the requirement because the implementation might do other things.

Now consider a requirement such as “the component shall not do D”. The implementation does satisfy the requirement, but not necessarily in a helpful way. Just because the component doesn’t do D, what should it do? Are behaviors A, B, and C all acceptable? What about behavior E?

In most cases it is clearer to name exactly the behaviors that are required, because that is unambiguous. One can write verification conditions to test exactly what is allowed.

Sometimes, however, one should write a negative requirement. If there is some behavior that really, truly must never happen, then writing a “shall not” requirement calls out that important condition, and a verification test can be designed to show that the system will not do the thing it isn’t supposed to. The negative requirement should usually be paired with a positive requirement that says what the system should do instead.

Safety and security properties often require stating a negative requirement, because these properties are fundamentally definitions of things that the system is to be designed not to do. I have not been able to imagine a way to write “a robot may not injure a human being” [Asimov50] as a positive requirement.

Verifying negative requirements is more complex than verifying positive requirements. See Section 14.4.

34.5.7 Avoid “it”

Avoid the word “it” and other non-specific pronouns or modifiers (“they”, “those”, “them”, “its”). Repeat the name of a thing involved in the property, even if that seems repetitive and wordy. An example:

This is better written:

Because the “it” in the first example is ambiguous: the word could refer to the mode or to the control system.

34.5.8 Avoid impossibly high bars

There are things that we want a system to do. When writing a requirement, it is tempting to write something like

Unfortunately, this three-year required property of the spacecraft is virtually impossible to meet (unless, maybe, the “spacecraft” is a large, inert chunk of rock). A spacecraft has many parts, operates in a difficult environment, and is built by fallible humans.

The problem with this requirement is that it sets a bar that is so high that no real spacecraft can meet it. The requirement does not allow for any off-nominal operation. It doesn’t allow for a spacecraft to have a temporary fault and then recover. It doesn’t allow for debris to impact the spacecraft. In fact, this requirement is met only when the spacecraft is perfect for those three years. Any real spacecraft will fail verification if it has a requirement like this.

This kind of requirement needs to be modified to something more realistic. There are many ways to do that. The NASA Systems Engineering Handbook has the rule that a requirement should specify “tolerances for qualitative/performance values (e.g., less than, greater than or equal to, plus or minus, 3 sigma root sum squares)” [NASA16, Appendix C].

Three common ways are:

• To place a probability on the requirement: function nominally with probability at least 95% for three years while on orbit
• Provide an availability or reliability measure on the requirement: function nominally at least 99.3% of the time for three years on orbit
• Provide a definition of “nominally”, either in other design documents or in additional requirements, that allows for the spacecraft occasionally to go into safe mode or otherwise be temporarily out of service

Of course, these are often combined.

34.5.9 Measurable conditions

The point of a requirement is that someone can determine whether an implementation complies with the statement in the requirement. Operationally, this means that a requirement can be verified (see the section on verification below).

One way to make a requirement measurable is to specify the condition quantitatively. For example, a spacecraft’s battery must be able to store at minimum X milliamp-hours. It’s not hard for a test engineer to see how to create a test to verify that the battery complies.

Other requirements, especially those that specify an action that should be taken under some condition, aren’t quantitative, but instead are measured by observing whether the required action is taken. The verification tests will involve either creating the condition under which the action is to occur or observing that the condition has occurred, and then observing that the required action has been taken. For this kind of requirement to be useful, a test engineer must be able to understand accurately the enabling condition and be able to create or detect that condition. The test engineer must also be able to understand the action that is supposed to occur, and detect that it has occurred. If the enabling condition or action can’t be detected, then the requirement is not readily measurable.

Requirements on low-level components are often easier to make measurable than requirements on high-level components. This is why high-level requirements are often verified by looking at requirements derived from the high-level requirement rather than by trying to construct a verification test directly on the high-level requirement.

34.5.10 Unverifiable conditions

When writing requirements for human-machine interaction or user interfaces, the underlying need is that a user can understand what the system is doing, and give it the right commands so that the system does what the user wants.

How would someone verify that the system as designed or implemented actually meets this objective? The statement is too vague actually to test.

There are multiple ways to address this issue.

First, one needs to break the objective up into a number of more-specific objectives. This often involves putting together a list of what it means to “understand what the system is doing”. This might involve:

• Perceiving what operating mode the system is in
• Observing fault conditions that will change operating behavior
• Observing how much fuel or electrical charge the system has remaining
• Interpreting the indications to understand the state of the system

And so on.

This breakdown is an improvement over the original desired objective, but the conditions are still not verifiable. As we will see in the later section on requirement derivation, these can be turned into high-level requirements that are broken down further, and the verification condition on these high-level requirements consists of, first, verifying all of the derived requirements, and then showing an argument that satisfying all the derived requirements shows that the high-level requirement is satisfied.

The derived requirements about “perceiving” or “observing” are themselves not verifiable: how does one verify that a person has observed, or can observe, some state of the system? This needs to be broken down into yet further, more specific requirements. For example,

Is a process, consisting of a chain:

If all these steps are satisfied and work correctly, then the person should be able to see the amount of fuel remaining.

Focus on the last two functions in the chain: that a person can see the indicator and that they can observe the indication. Seeing the indicator can be in turn broken down into further requirements, primarily on the physical structure around the person. For example, some of these might be:

• The person observing must have an unobstructed line of sight to the indicator
• The indicator must be visible in all the lighting conditions that the person will experience
• The indicator must cover a large enough portion of the person’s field of view that they can see the details of what the indicator is showing

There is some prerequisite information needed to verify these examples. For example, what range of sizes will the users be? In order to check for unobstructed line of sight, one must know where the user’s head will be. What visual acuity or color perception abilities are required of the users? A color blind user will not be able to perceive some color differences that might be used to convey necessary information. What expectations will a user bring to the task? If a user is socially conditioned that green means good and red means bad or stop, using different colors to indicate good or stop will be hard for a user to interpret.

How would one go about verifying these requirements? There are multiple techniques that will help—and usually the techniques must be used together to really check whether a requirement is satisfied. These techniques are a combination of analysis using models and real-world measurement.

• Analysis of CAD models or physical models of the environment. If we know where the indicator will be placed, what else is around it, and where the user’s head can be, one can determine if something will obstruct the line of sight. Similarly, building a model of all of the anticipated lighting conditions can allow one to find the lighting situations that are likely to be difficult for perception, such as having the sun in a user’s eyes when they try to look at the indicator, or full darkness.
• Standards. There are many standards for color, contrast, visual size of an indicator, and many other aspects of user interface components. Meeting a standard may be sufficient for determining whether the requirement is met.
• Experimentation. Build a mockup of the environment where the indicator and user will be, put users in that mockup, and measure how well they do at perception tasks.

The experimental approaches are often the most expensive in time and money, but they are the gold standard for verifying a human interface requirement. Conforming to standards can help address expectations that users will bring to tasks.

In summary, there are several tools for addressing requirements that are too vague or complex to verify:

• Writing a high-level requirement, breaking it down into multiple derived requirements, and showing that satisfying those derived requirements means that the high-level requirement is satisfied
• Consider the chain of functions that leads from the basic states of the system or component to the desired outcomes, and use those functions to guide a breakdown into lower-level requirements
• Determine how to perform verification on the lowest-level requirements, using a combination of analysis, standards, and experimentation

XXX revisit this section to bring it into line with the Leveson viewpoint on user interaction as control

34.5.11 Detail appropriate to the level

Requirements should be written as a description of what one sees in a component when looking at it from the outside—a black box view. A good requirement does not go into how the feature or behavior is implemented inside the black box.

Put another way, the requirements for a component are documentation of how the component fits into the system around it. If component A is part of a larger component B, the requirements on A document what the implementation of B needs for A to do its part correctly. If components C and D are peers, the requirements document what they will need from each other for both to do their job.

This matter connects directly to requirements derivation from component to subcomponent, which is discussed in the next section.

There are four reasons to follow this rule.

1. Requirements aren’t the only specification of the system. There are design documents whose job is to document how a component will be implemented internally.
2. Many requirements are written before a component’s internal implementation is understood. The requirements serve as a record that the component designer can come back to to make sure they have designed or built a component that meets the needs of the components or users that will interact with the component.
3. Things change. Components get redesigned. If a component’s features don’t change but its implementation does, the requirements defining the component shouldn’t change.
4. Saying what a component is supposed to do leaves room to document the thinking or the rationale that led from the external what to the internal design of how the component provides the whats. This helps others who come along much later to understand the system—in particular, it helps when a requirement needs to change and a new person has to work out what effects that change in requirement will have on an implementation.

It is tempting to skip right to the details of how a component is built. Don’t do it; provide other people the benefit of your understanding of the problem, not just the final design answer.

34.6 Requirement derivation

XXX revisit this to bring it in line with system model terms

No requirement stands entirely on its own. Almost all requirements have some reason that they have been included in a system, starting with: this requirement is necessary so that the system meets some objective. In lower-level components, the reason often is: this requirement is necessary so that this component provides some feature that other components depend on.

These are examples of requirement derivation. Derivation encodes the relationship between requirements.

Almost all requirements are derived from other requirements, and the requirements in a system must keep track of how one requirement leads to another, or how one is dependent upon another.

There are several kinds of relationships that people record. Some of these are:

• A lower-level component has a function that a higher-level component will use
• One requirement within a component implies other requirements on that same component
• A component passes on a general requirement to its subcomponents
• Two peer components will be interacting, and depend on each other’s functions

Let’s look at each of these kinds of derivation.

34.6.1 Subcomponents providing features for parent

A parent component has a requirement that the component provide some feature. The requirement in the parent specifies what the parent must do, but does not specify how to implement that feature. The design of the parent component, and later, the implementation, document how the parent component will satisfy that requirement.

When the designer decides on the implementation, they will decide (among other things) how the parent component will use subcomponents to implement the feature. These decisions create requirements on the subcomponents so that they provide the features that the parent component will use.

The reason for these requirements on subcomponents is that they are necessary to satisfy the requirement on the parent component. A derivation relationship between the parent requirement and the subcomponent requirements documents why the subcomponents have the requirements they do.

Consider a spacecraft example. The spacecraft as a whole has a requirement that it be able to point at a ground location, with some number of degrees of accuracy. To implement that feature, the spacecraft designer chooses to use the spacecraft’s attitude control system to point the spacecraft toward a ground location, and then slowly rotate the spacecraft as it passes over the ground location. The parent component—the spacecraft—has the high-level requirements for what it needs to do. The subcomponent—the attitude control system—must be able to slew accurately to an initial pointing vector, and then be able to slew slowly and accurately until the spacecraft is done with an observation. The slewing accuracy and speed are the derived requirements on the attitude control system.

The process continues recursively. The attitude control system designer decides to use reaction wheels as the primary attitude control mechanism. The requirements for slewing accuracy and speed create requirements on the reaction wheels for how quickly or slowly they can turn the spacecraft.

34.6.2 Internal derivation

Some components will have a requirement that specifies a very high-level capability the component must provide. For example, in a section on disposing of a component that is being discarded:

There are several ways this requirement could be met: destroying the retired component in house, crashing the component into the atmosphere or ground in a way that will assure the component is destroyed, or erasing the data on the component before giving the component to an outside entity for recycling.

Whatever the implementation decision is, it creates more requirements on the component, and those requirements derive from the decision on how to satisfy the requirement on protecting confidential information. If, for example, the implementation decision is to recycle a retired part, then this might lead to requirements like:

In some organizations, the practice is only to record derivation from one component to another. Sometimes that works out; in the example, the requirement for an erasure command could be on a command handling subcomponent, and the erasure requirement could be on a memory component. However, some components do not break down into subcomponents easily—for example, when the component is being implemented by an outside vendor. In other cases, it is simply clearer to document the implementation requirements for the component directly and then passing the requirements through to subcomponents, so that a user can see the totality of the functional interface to the component in one place rather than having to search through subcomponents for something they don’t know exists.

34.6.3 Pass through

External objectives and standards often impose general requirements on “all components of type X”, or the like. For example, an automobile might have a requirement that all electronic components function nominally across a temperature range of -40º C to +125º C. (See the section on Sets as subjects below for more on this.)

This requirement can be placed on the automobile as a whole; the requirement might read

If the automobile includes engine, braking system, and entertainment systems as parts, the temperature range requirement can be passed down to those subcomponents:

But the entertainment system, which is not safety critical and operates in the more benign environment of the passenger cabin, might have the requirement:

In these examples, the general requirement is copied down into lower-level subcomponents until it reaches some component (such as the braking controller in the example) that does not have further subcomponents. Sometimes the requirement is copied verbatim, just changing the scope of the subject; other times, some component will have a variant on the general requirement.

This kind of derivation is sometimes referred to as allocating requirements to subcomponents.

34.6.4 Mutual dependency

Sometimes two components are peers of each other, and need to interact. A fuel tank provides fuel to an engine; a spacecraft communicates with a ground station to send telemetry and receive commands; client and server applications send messages to each other.

These interactions involve requirements on each of the components involved, showing how the components support each other. The fuel tank must send fuel; the engine must consume fuel. The spacecraft must be able to communicate with the ground station; the ground station must be able to communicate with the spacecraft.

This leads to pairs of requirements that record this mutual dependency. At a high level,

and

These two requirements should show a two-way relationship with each other. (Formally, this introduces a cycle in the derivation graph.)

34.6.5 Using derivation

Derivation shows how requirements are related to each other.

Systems engineers use the record of these relationships for several tasks.

• Determining whether a design of the system can satisfy its top-level objectives, and encoding how a system’s design satisfies those top-level objectives.
• Determining whether a particular requirement is actually needed (and why).
• Tracking down the effects of making a change in one piece of a system on the rest of the system.

A derivation relationship between requirements on two different components helps to document the implementation approach for meeting a higher-level requirement. When a designer looks at the high-level requirement, they can see what features are used to implement the high-level requirement. The lower level requirements and their rationale allow the designer to see the argument that the implementation will be sufficient to meet the high-level requirement. This makes the design rationale available to people who didn’t create the design in the first place, but need to understand it to evaluate it or to make changes.

The section on analyzing requirements, below, goes into more detail on how one can look at the requirement derivation relationships to evaluate completeness or sufficiency, to argue whether low-level features are actually necessary, and to trace out the effects of making a change in requirements.

34.6.6 Viewing derivation

There are two ways that a user should be able to see derivation relationships. First, when looking at any one requirement, the user should be able to see what requirements this one is derived from directly, and what requirements derive directly from this one.

Good requirement management tools will also provide a view of the graph that shows derivation graphically. Derivation relationships can be viewed as a graph, as a way to see multiple levels of derivation. The graph is typically mostly a tree or DAG, but there are legitimate reasons that the graph will sometimes have cycles (between peer components, for example).

Here is an example showing how a top-level requirement is the source for a number of other requirements.

34.7 Advanced requirements

All the requirements discussed so far are simple requirements. Simple requirements have a single, clearly specified subject component. Each simple requirement expresses one property about that subject that must be true.

Simple requirements are not sufficient to express every need that real systems encounter. There are two that we have seen many times: requirements on sets of components, and requirements for standards.

34.7.1 Sets as subjects

Consider a system where all code is expected to adhere to a published coding standard. The implied requirement does not apply to any single component; it applies to all of them that include software.

This expectation can be written as a top-level requirement on the system as a whole:

The subject of this requirement is the set of all software components in the system. The property is that their implementation adheres to the named coding standard.

This kind of requirement is placed on the top-level system, and then each first-level subcomponent includes a derived requirement that propagates the requirement downward:

On a component Y that has software as part of its implementation then has:

If component Y has subcomponents, Y should also have a second requirement that continues to pass the requirement down to Y’s subcomponents.

This is an example of a general technique:

• At some component, levy a requirement that is expected to apply to some subset of the subcomponents of that component
• Pass the requirement down from component to subcomponent by giving the subcomponent a derived requirement that is nearly a copy of the original, except for limiting the set of components
• When the derivation reaches a component to which the need applies, write a simplified requirement that names only that one component as its subject

34.7.2 Writing for standards

Many texts on requirements approach the subject from an assumption that there is one system being built: these are the requirements for System X. System X will be built in its entirety as specified; any and all requirements must be satisfied.

Writing standards is a different problem. A standard is specifying requirements on multiple hypothetical systems that may exist at some point. Those systems will not be identical, but the systems that adhere to the standards must adhere to the requirements in the standard.

Standards often provide options. The standard has a set of optional features. If the system chooses to implement those features, the features must conform to the standard. However, the system does not have to implement those features. This means that the system does not have to satisfy every requirement in the standard.

Some standards also present best practices. For some feature, it is recommended that the feature conforms to a part of the standard, but it is not absolutely required to do so.

The vocabulary of “shall” or “must” does not accommodate these situations well. The Internet Engineering Task Force (IETF) has defined a richer set of requirement modes. For example:

The words used to indicate these more complex conditions must be defined just as carefully as “must” or “shall”, and must be used consistently.

34.8 Analyzing requirements

Many people think of requirements only as a contract for guiding implementation and a checklist for performing verification tests later. However, requirements—along with other specifications—are useful in themselves for helping build a design and making sure the design is good.

There are three kinds of analysis that systems engineers do on the requirements themselves:

1. Ensuring that the requirements (and specification) are complete
2. Ensuring that the design is minimal, meaning that the design only contains features that it actually needs and nothing extraneous
3. Ensuring that the requirements are consistent
4. Understanding the effects of making a change to one part of a system design

These are all analyses that should be done on the specifications of a system, including the requirements, and not delayed until implementation. Some of these tasks are easier to perform on the abstracted and simplified view of the system that specifications give. Performing these tasks before implementation will reduce the amount of re-implementation needed when one finds that the requirements aren’t sufficient or minimal.

34.8.1 Complete design

The expectation is that if a system is built to conform to its specification, including requirements, that the system will do the job that its users need and do it correctly. (Of course, this assumes that the top-level specifications are themselves a correct and complete record of the users’ objectives; we discuss this more in the section on validating requirements below.)

To meet this expectation, the system’s requirements need to be complete and correct. This means that when one looks at any given top-level requirement, one can trace out the features on other components that will be used to implement the requirement and argue that those features will combine correctly to produce the desired result.

There are two parts to this analysis:

• At each level of derivation, are the derived requirements complete for their parent? Is there a rationale that shows how these derived requirements are necessary and sufficient to implement the parent?
• Are the leaf requirements concrete, implementable, and verifiable?

Having tools that allow one to view parts of the derivation graph in visual, graphical form is invaluable to performing this analysis.

Consider an example. A UAV (drone) is supposed to receive and process commands from an operator on the ground. This leads to requirements:

These requirements are not complete, because they leave out a critical step: when a command is sent from the ground operator to the UAV, the message first goes to the transceiver. The receiver extracts the message, and then sends the message to the command and data handling component. The example omits the part about the transceiver and command handler passing information to each other. This means that one could build an aircraft that had a radio and had a flight computer, but the two would never talk to each other. Obviously, the UAV would not be acting on commands with that design.

This leads to a more complete set of requirements:

In the example, the communication between the transceiver and command handling components should be documented in some other specification for the UAV, perhaps an activity diagram showing how commands flow through components. The requirements then need to be checked against these other parts of the specification to make sure that all of the functions in each of the steps are reflected in the functions each component is required to implement.

Sometimes determining whether a set of requirements is complete or not will require further analyses. As a simple example, the maximum mass for an aircraft might be X kg. Making sure that the aircraft’s overall mass comes in under that limit means enumerating all the components in the aircraft that have mass, adding up their mass, and determining that the result is below X kg. For that analysis to be complete, it cannot leave out, say, the mass of the motors; all components must be considered.

As a more complex example, a system might have a maximum acceptable failure rate target. Being able to argue that the system is reliable enough involves performing a fault tree analysis, enumerating all the ways that failures in components can lead to system failures. The analysis cannot leave out components and be complete; nor can it leave out some failure modes of some of those components.

Checking whether the design is complete is not a simple task that can be performed just by inspecting the graph of requirements. The analysis is helped by being able to see the requirements, but it requires imagination and effort to actually check the result.

XXX Sidebar: relationship to Goal Structuring Notation and safety cases

34.8.2 Minimal design

Every feature and every requirement on a component should have a reason for being there.

At the top level, for the system as a whole, only features that address customer needs or business objectives should be included. At lower levels, the only requirements that should be placed on components should be ones that are actually needed to make the system work properly—meaning the system meets those top-level objectives.

34.8.2.1 Tracking the purpose for every requirement

The derivation relationships between requirements encode the reasons for a requirement to exist. This leads to a condition that should hold across all requirements:

This is straightforward to check using the derivation graph: every requirement should derive from at least one parent requirement, and it should be possible to trace upward through the derivations to reach a customer or business objective.

Often while requirements are being developed, a requirement will be placed on some component without setting up the derivation. This requirement will not have a parent, and so the checking method will flag it. But what to do then?

In most cases, there was a good reason that someone wrote that component requirement. When one finds a requirement that is not documented as supporting some higher-level reason, it is worth exploring why that requirement is valuable. In some cases, the parent requirement(s) are present, and the requirement just needs to be linked to them. In other cases, the requirement can be a clue that there is some higher-level principle that the writer had in mind, and that higher-level principle should be added into the requirements higher up in the system.

For example, consider a data storage component where an engineer placed a requirement that all data be stored in an encrypted form. As written, that requirement doesn’t derive from any other requirement. But why did the engineer believe that encryption was necessary?

One answer is that encryption isn’t necessary. In that case the encryption requirement can be removed. Another answer is that the engineer wrote that requirement because they believed that the component would be storing confidential data that should be protected against disclosure. In that case, it is worth checking: does the system have requirements—or business objectives—about protecting confidential data? If not, then this exercise will have found a topic that has not been adequately addressed, and new requirements need to be added to make a correct specification. Those requirements should be added throughout the system, and the requirement we started with should show that it derives from those new features.

Many such requirements result from external standards that are supposed to be met, such as regulatory, safety, or security standards. Those standards should be included in the external objectives for the system, their requirements should flow down through the system to the components where the standards apply. This produces a record of how the system’s design complies with those standards.

34.8.2.2 Finding unnecessary requirements

Some requirements that show how they are derived from some parent requirement are still not actually necessary.

There is no simple, mechanical way to find these unnecessary requirements. However, the analysis used to determine whether a collection of requirements is complete is also useful for finding these unneeded requirements.

Consider this example:

The requirement about encryption is not actually needed for the system in question. That is because the connection between the transceiver and command handling components is physically contained within the UAV, and the physical encapsulation provides enough security to protect the messages passing between the two. The encryption requirement can be removed with no loss of capability.

However, in this example, the engineer who wrote the encryption requirement had a good idea but expressed it wrongly. The engineer understood that the integrity of communication between the two components was important; a command that was properly received but garbled in being sent to the command handling component could be a problem. The presence of the encryption requirement should be replaced by a less costly requirement, that the channel must protect the messages it carries against corruption.

34.8.3 Consistency

Consistency in a body of requirements is when the requirements don’t contradict each other. If requirements do contradict each other, the system as specified isn’t implementable and the specification needs to be fixed.

Broadly speaking, there are three kinds of consistency that one should check:

1. Consistency among requirements for one component
2. Consistency between requirements on either side of an interface between two or more components
3. Consistency between requirements on a higher-level component and the requirements on subcomponents that should show how the higher-level requirements will be implemented

As long as requirements are written as text, and not in a formal notation, consistency checking will be manual. It involves reading through each requirement, finding other requirements that address related topics, and checking that they are consistent with each other.

Some inconsistencies are fairly easy to detect. If one requirement says component X shall be blue and another says component X shall be red, it’s obvious—one must just read through all the requirements on component X and see that two requirements both deal with the color property and they say opposing things.

Other inconsistencies are harder to spot because they do not use the same language in the properties they are specifying. As an example, one requirement might say component X shall use encryption algorithm Y while another requirement says component X shall use protocol standard Z. If protocol standard Z allows encryption algorithm Y, this is fine. But if the standard does not allow that particular encryption algorithm (perhaps because the algorithm is outdated and no longer considered secure enough) then there is an inconsistency.

Another class of inconsistency comes from the states a component can take on. Elsewhere in the specification of a component, there should be a definition of the state machine that the component is supposed to follow. The requirements translate that state machine into individual actions that the component is expected to take in response to particular inputs. It is easy—especially when editing or updating the component’s specification—to have two requirements: when condition A occurs, component X must transition to state Y and when condition A occurs, component X must transition to state Z. The inconsistency can be more subtle, such as leaving out some transition, or using inconsistent definitions of the condition that causes the transition. This class of problem can be addressed by having a single, clear definition of the state machine the component is expected to follow, and then checking the requirements against the state machine.

Finally, another class of inconsistency that can be hard to detect has to do with timing. Two requirements can impose timing constraints that cannot both be satisfied. For example:

There is no way for component X to meet the timing requirements given the order that events must occur. Building a timing model of the component in question, and performing a timing feasibility analysis using that model, can help find this kind of inconsistency.

This is by no means an exhaustive list of the kinds of inconsistency one must look for.

34.8.4 Effects of changes

Systems change. This can happen because customer needs change, or because technology changes, or because someone has found a better design for part of the system. A good development process supports constant evolution and change of the design and implementation of a system.

Not every change that is proposed will be performed. When someone proposes a change, someone else will analyze the proposal to determine the effects of the change. Based on this analysis, people may decide to go ahead, postpone the change, or not make the change.

The analysis must accurately determine:

• What components are affected by the change? Is it just one component, or do effects ripple through much of the system?
• What things will need to change as a result? Component designs, specifications, safety analyses, regulatory filings?
• How extensive or expensive will those changes be?

This analysis makes use of all the specifications in the system, but requirements are a major contributor. In particular, the derivation relationships help show how component features depend on each other, and thus help guide an analysis of how far some change will spread.

34.8.4.1 Effects of top-level changes

Top-level changes include adding a new feature to the system, removing a desired feature, or changing a standard or other external source of requirements.

If the change changes a top-level requirement, look at the derived requirements from that changed requirement and see if the derived requirements are still necessary and sufficient to satisfy the newly-changed requirement. If they are, then no further action is needed. If they are not, then the derived requirements must be revised, possibly adding or removing some of them. The process then needs to repeat with these changed derived requirements. If the change affects a requirement that supports a different top-level requirement, then one must check that the other top-level requirement is still satisfied by the changed derived requirements.

If the change adds a new top-level requirement, work out what derived requirements are necessary and sufficient to satisfy the new requirement. Look for lower-level requirements that already exist that can also support the new requirement. This may involve a change in design, not just requirements; this will cause more changes to propagate out.

If the change removes a top-level requirement, see if any lower-level derived requirements are no longer needed or can be relaxed. If so, work downwards to propagate the effects of those changes.

34.8.4.2 Effects of lower-level changes

Many more changes will come to lower-level components in the system. There are many reasons this can happen: because people have found that a design in process is infeasible or too costly; because a vendor’s part specification or availability has changed; or because someone has found a better design for some lower-level component.

Evaluating a lower-level change involves all the checks for a top-level change above, along with the need to see how the change will affect higher-level requirements. Will the change leave the higher-level requirement unsatisfied? Will this change make some other sibling requirement redundant (that is, the parent is satisfied without the sibling)?

Tracking down these effects is much easier if the derivation relationships among requirements are accurate.

34.8.4.3 Tools

Good tools help the process of evaluating changes. There are three features in particular to look for:

1. The ability to create an independent working version of the requirements, in order to try out changes before committing them to a baseline. The ability to see what has changed between the baseline and the working version and selectively merge changes into the baseline allow reviewers to understand the whole effects of the change and to accurately accept the changes.
2. A feature to mark some requirements as potentially changing and others as needing evaluation. This feature helps ensure that the change evaluation does not miss some important change.
3. The ability to record a rationale for a derivation relationship between requirements helps the people evaluating changes determine why a set of derived requirements was considered necessary and sufficient.

34.9 Validating requirements

XXX rewrite this to bring into line with introductory language on deriving verification

Validation is the process of determining whether a set of requirements accurately reflects the needs of the system. This can mean that the system will meet customer needs, or mission needs, or other external objectives.

It is important to keep validation separate from verification, which is discussed below. Validation is about seeing if the requirements (and the rest of the specification) is an accurate reflection of external needs. Verification is about seeing if the implementation is an accurate reflection of requirements. (Some software engineering texts focus validation on consistency, completeness, and similar properties. Systems engineering has generally kept those kinds of checks separate from validating customer or mission satisfaction.)

The validation process starts with checking the system objectives, business objectives, security and safety objectives, and regulatory objectives to see if they are an accurate reflection of the customer or mission needs. Presumably appropriate care has been taken while these objectives are being gathered and written down, but mission understandings or desires change over time and an independent check on the objectives will help avoid having problems be discovered late, when it is expensive to make changes.

At the top level, one should check:

• Is there anything in the top level that does not come from what the stakeholders want?
• Is there anything the stakeholders want that is missing?
• Are all the regulatory needs addressed?

At lower levels, one is checking whether the derived requirements from a parent are necessary and sufficient. The analyses for complete and minimal design, discussed above, cover those checks.

There are many different ways to validate a system’s specifications. They generally fall into two groups: analysis and simulation.

XXX improve language: analysis as formal method vs review as informal

Validation by analysis involves people reviewing the requirements and using their judgment to check the specifications. This can involve performing joint reviews with stakeholders so that they can check the requirements.

Validation by simulation involves stakeholders somehow seeing a model of the system in action. There are many ways to do this. Stakeholders can be invited to define some scenarios that represent how they will use the system, and then try out those scenarios using a model of the system. Some ways we have done this include:

• A desk or whiteboard exercise, where one or more engineers act out the system’s behavior and some stakeholders talk through what they would want to happen.
• Building a simple simulation model of parts of the system and letting the stakeholders try out their scenarios with the simulation.
• Building prototypes of parts of the system and having the stakeholders try out the parts of their scenarios using the prototype. This can be combined with a desk exercise for parts of the system that aren’t prototyped.

These validation exercises should be completed and the stakeholders should concur that the specifications are correct before one baselines the specifications, including requirements.

34.9.1 Connecting requirements and implementation artifacts

People must be able to navigate from a requirement to its associated implementation artifacts and vice versa. The people implementing a part of a system according to requirements need to be able to quickly and accurately find the requirements that they need to comply with. In the other direction, the people verifying requirements must be able to find the artifact or artifacts that implement a particular requirement.

The approach to organizing systems artifacts that I advocate here, which organizes many systems work around a hierarchical component breakdown structure, is designed to meet this need conveniently. The set of requirements that apply to some component are implicitly connected to other specifications and the implementation of that component because they are all organized by the same component names and identifiers.

One can also explicitly label artifacts with component identifiers or requirement ids. For example, verification test specifications are associated with specific requirements, so the test specification needs to be labeled with the requirement ids that it applies to.

34.10 Verification

Verification is the process of showing that the implementation of the system, or parts of it, complies with the requirements.

Verification involves gathering evidence that every requirement is satisfied by the implementation.

There are four general methods used to verify the implementation’s compliance:

• Inspection: review of the implementation (source code, board designs) to manually check that the implementation complies
• Test: define and perform tests on the implementation that check compliance. Many requirements require a number of tests in order to show that the implementation complies given all reasonable inputs or environment conditions
• Demonstration: show the implementation working as defined
• Analysis: perform a static analysis on the implementation to gather formal evidence that the implementation complies

Inspection is verification by having people review parts of the implementation to check that it complies with a requirement. The inspection review should be performed by people who did not implement that part of the system, so that the reviewers are not misguided by preconceptions (“I’m sure I implemented this correctly”).

Some inspections are particularly simple. Consider a high-level requirement that is the source for a few lower-level requirements. In many cases, the high-level requirement is satisfied when the lower-level derived requirements are all satisfied. In these cases inspection becomes a simple matter of checking that the derived requirements are all satisfied. The rationale associated with the derivation or with the high-level requirement should indicate when this situation applies.

Test and demonstration are similar. Testing is generally more exhaustive, and necessary lower-level components. A single electronic component, for example, might be operated across all the specified thermal, vibration, and atmospheric environments it must handle. Demonstration is less exhaustive, and used to verify top-level system objectives. A prototype spacecraft radio transceiver might demonstrate that it can communicate with ground stations from a similar orbit to where the final spacecraft system will operate.

Some requirements cannot effectively be verified by test or demonstration, and must be verified using analysis. This occurs when one is verifying a negative condition: the verification must show that the system will not perform some action or be in some condition at any time. Providing evidence of the absence of some condition is a long-standing scientific and engineering problem because proving the presence of some condition is relatively easy—demonstrate it happens in one case and that’s sufficient—but showing absence often requires exhaustive search. These verification problems often arise in safety and security requirements, where unsafe failures must be rare (e.g. no more than once in 10⁹ operating hours) or a system must resist a class of attacks (showing that no attack of that class will succeed).

Each requirement should have an associated verification specification. The specification should lay out what steps must be taken to determine whether the implementation is correct or not. A verification specification is often complex—many pages of documentation for a three-line requirement.

Verification status is a measure of how well the implementation matches the specification, including requirements. In practice this means how well a version of the implementation complies with a version of the specification, as both implementation and specification evolve over time. This means that, during design or implementation, there is no one single “verification status” that can be tracked: with each new update to the implementation, the verification status changes. Some practitioners and tools make the mistake of tracking verification status only in terms of requirements: which requirements have been satisfied by the implementation? This leads to project management errors when a change is made to the implementation that improves the implementation in one area but causes other parts of the system to go out of compliance—a common occurrence while in the middle of implementation using iterative approaches.

34.11 Limitations of requirements

Requirements have limitations. Writing a good specification for a system means understanding these limitations and addressing them in one way or another.

One limitation is that requirements are written in natural language. Human language is notoriously difficult for pinning down precise meanings, even within a single group of people. Specifications, including requirements, are used to communicate between different groups of people with different outlooks, experiences, and jargon. This makes it hard to write requirements that will be interpreted the same way by all of the people involved.

The limitation of natural language can be partly mitigated using a couple of techniques. One is to maintain a glossary that defines words or phrases that have specific meanings in the specification beyond common understanding. The second is through social cohesion: having enough people from different groups interacting and discussing the system so that they evolve a common understanding of the meanings of things.

Precision is another limitation. Some specifications can be clear and simple in mathematical notation, while they are hard to follow in prose. (Consider expressing Newton’s law of gravitation as an equation versus in prose.)

A third limitation comes from requirements being single statements. Sometimes the specification needs to encode a complex, multistep activity. Each of the steps might be encoded as an individual requirement, but it is awkward and hard to understand. Sometimes the better answer is to write part of the specification in a different form—a flowchart, a state machine, or a set of equations.

As a result, requirements are only one part of the total specification. They cannot do the entire job of recording the full specification of the artifact in question—but they are often the most flexible way to organize most of the specifications. Be prepared to supplement textual requirements with other kinds of specification to get the whole job done.

34.12 Working with requirements

This chapter has mostly covered what requirements are. This section touches on what one does with them and how they evolve over time.

Requirements will change continuously over the life of a project. The rate of change will be high at the project’s beginning, when the team is trying to sort out what the system should be. The rate of change will increase after the high-level system purpose is sorted out and as the design work proceeds in parallel on different components in the system. The rate will taper off as the design and implementation become more mature, with occasional bumps as people find problems with the specifications, or as stakeholders request changes. Ideally the rate will reach zero when the system is ready to go operational, but even while in use people will find changes they would like to make.

Detailed requirements are expensive to develop and maintain. They encapsulate the complexity of how all the parts of a system are interconnected. They require effort to develop in the first place, involving checking for consistency and feasibility across large parts of the system. Changes later involve even more effort, especially if the changes involve reorganizing specifications that have already been developed.

This leads to a tension: changes will always happen, especially with modern, flexible systems, but the cost incentivizes developing all the requirements at once and then freezing them to minimize the cost of change.

This tension is unavoidable, but there are things one can do to reduce the difficulty.

• Don’t start finalizing requirements too early. Work with a sketch version of the main points while exploring the concept.
• Develop detailed requirements for a part of the system hand in hand with prototyping the components that make up that part of the system. Most teams learn a lot from prototyping exercises; what they learn translates into design changes and in turn into requirements.
• Start the processes for communicating about specification changes early in a team’s development, so that people have developed the habit of talking to each other about changes that might affect others before major, disruptive changes come along.
• Use the right tools that help the team coordinate developing and changing the requirements.

34.12.1 Supporting the life cycle

The requirements for a system—and indeed all the specifications for the system—grow and evolve over time. The times and ways when requirements change depends on the development process a project is using. However, all these processes share some tasks in common.

34.12.2 Who works with requirements

Many people generate or use requirements during the lifetime of a project. These include:

• The product or mission managers who define the system’s objectives and review and approve the top-level system requirements
• The engineers who develop, revise, and maintain the requirements
• Change management people who oversee the process to baseline and release consistent, committed-to versions of specifications
• Designers and implementers who build components following the specifications
• Engineers who design how different requirements will be verified
• People who verify implementations against the requirements
• Managers who track the progress of implementing requirements and the progress of verifying components
• Vendor managers who oversee the process of writing requests for proposals, evaluate vendor offerings, and oversee the acceptance verification of artifacts from vendors
• Vendors who design and build components according to requirements
• Contract specialists who incorporate requirements into vendor contracts
• Technical writers who develop operational and training manuals based on the requirements
• System operators who use the requirements to understand how the system works in operation

34.13 Tools

The right tools make working with requirements much easier and more accurate. However, different requirements management tools are designed to support different styles of requirement writing and use, so you need to choose tools that match how you will write, organize, and use requirements.

Here are some questions that can help you evaluate requirements management tools.

• Do the tools support the writing style and requirements organization strategy you use?
  • If your project is using hierarchically-organized requirements, does the tool support that?
  • Do the tools provide a way to organize requirements for one component into sections?
  • Do the tools support organizing requirements in the way your project organizes them? If you organize requirements by component, do the tools support that?
• Do the tools provide ways to trace top-level requirements from external sources, like policies or standards documents, that are the source for those requirements?
• Do the tools support tracing the derivation relationships among requirements?
  • Do the tools allow derivation among requirements within one component?
  • Do the tools allow mutual relationships between two components, when those two components have mutual dependency?
  • Do the tools provide ways to visualize the graph of derivation relationships?
  • Do the tools allow one to record a rationale for derivation relationships, in addition to rationales for individual requirements?
• Do the tools support the relationship between requirements and implementation artifacts or other documents?
  • Do the tools provide a stable identifier for requirements?
• Do the tools support reviews and baselining, or other versioning control mechanisms?
  • Do the tools support the ability to create a non-baselined, work-in-progress version of some requirements?
  • Do the tools provide a way to see the differences between a baselined version of requirements and a work-in-progress version?
  • Do the tools enforce a review and approval process?
  • Do the tools help notify those who are affected by a change and who thus should provide a review?
• Do the requirements tools integrate with other tools that support verification?
  • If the requirements change, do the tools make it clear what requirements must be verified anew?
  • If the implementation evolves, do the tools allow for verification results to be associated with a version of the implementation and not just with the requirements?

People will use the requirements management tools to perform a number of tasks. You should evaluate how well requirements tools support these activities.

• Writing new requirements: being able to quickly sketch out a set of requirements for some new component or some new function, and then fill in the details to complete a first draft of the new requirements. Ease of writing and editing is a key complaint about many requirements management tools.
• Reviewing and updating requirements: when someone requests a change to the system design or objectives, being able to find what things need to change and then being able to revise the requirements conveniently.
• Managing review, approval, and baselines: at some milestones, reviewers will need to see what requirements they should review, support them in the process of discussing the requirements, and tracking issues that are found until they are resolved. With some milestones, a version of the requirements will need to be approved (or not) and baselined.
• Performing requirements analysis: people will need to perform the analyses listed in the Analyzing requirements section, including checking that the design is complete, minimal, and consistent.
• Validation: people will need to review the requirements against the original mission or project objectives to see if the system will meet stakeholder needs. The reviewers are generally going to be independent reviewers who have not been using the requirements tools along the way. The tools need to support the reviewers determining whether the requirements are satisfactory, and tracking issues that are found.
• Verification: some people will need to develop tests or analyses to perform verification. Other people will then need to perform verification on system implementation artifacts, and track the status of how well the implementation is complying with requirements.

Chapter 35: Models

Chapter 36: Interface definitions

Part VIII: Design

• design practice XXX define this
• safety design and analysis
• security design and analysis
• design checking

Chapter 37: Design introduction

37.1 Purpose

Previous chapters introduced how to work out what a system or a component should do, by determining what the objectives are for it and then turning those objectives into a specification.

The next step is to design the system or component that will fill those needs.

A design for a component provides a simplified model of how the component will achieve the behaviors, qualities, and structure laid out in its specification. The design is not the full details of how it will achieve those things, or a detailed implementation. The design is a plan for how the component will be built, at a high level; it records the high-level decisions about how the component will be implemented without actually being the implementation.

“Design” is an activity that lacks sharp boundaries from other development activities. On the one hand, it responds to the objectives and specifications that have been developed for the thing being built; on the other hand, the act of designing usually reveals gaps in the specifications that lead to feedback that causes people to update the specification. Specification and design proceed recursively as a system is built, where the act of designing one component leads to writing specifications for its subcomponents.

“Design” also lacks a distinct boundary with “implementation”. Indeed, the boundary between the two varies by convention in different disciplines.

• In electrical engineering, a “board design” is a detailed mechanical design, choice of components, and layout of components and circuits—what one might classify as implementation. A higher-level design for a board would include things like the most important components (e.g. chip, SOC, discrete electronic components) that will be used, the kinds of interfaces, connectors, or cables that will interface the board to others, along with the record of a rough physical layout that shows that an implementation is likely possible. This higher-level design guides the board designer to create a board design that is ready to be assembled.
• In software, a design typically consists of high-level decisions such as where the software will be deployed, the structure of how subcomponents communicate with each other and with outside components, the state and operations that are to be implemented in the component, and behaviors it will perform. This design leads to a software engineer writing the specific code that implements the design.
• In architecture or mechanical engineering, a design records the general decisions about the shape or layout of the structure being built, how it connects to other components, and key design choices like materials or colors. This leads to detailed mechanical drawings that incorporate all the supporting details needed to build the structure.

37.1.1 Defining “design”

Given the diversity of ways the word “design” is used, I will define what I mean by the term in general.

A design is:

• A high-level record of the decisions made about the structure, state, and behaviors of a component or system, and how it meets its specification
• A record of how the system or component connects to things outside itself, including physical connections, data or control, or energy transfer
• A record of how the system or component breaks down into subcomponents, and the roles or specification of each
• A guide to the reviewers and implementers that helps them understand the component’s structure and function
• Simpler than the detailed implementation of the component, so that the design explains the implementation
• An abstraction of the implementation that allows one to analyze the component (for safety, security, performance, or meeting objectives) at a reasonable level of effort, when doing the analysis using a detailed implementation would be too complex

A design is not:

• Detailed enough directly to support assembling and operating the component or system
• The specification, which records properties of a component without saying how it should be designed to provide those properties
• The implementation, which is has enough details that the actual component or system can be assembled, integrated, tested, and deployed

In some projects I have used the term “design model” for the design, to emphasize that the design is a simplification and explanation of the most important aspects of the component’s implementation.

37.1.2 Contents of design

There are several kinds of information that should be recorded in a design.

• The components that make up the system, with names or identifiers for each. This can be thought of as the catalog or table of contents for the system. Every component in the system should be included, whether it is built from scratch for this system or brought in from some supplier. Having a list and names allows people to identify what they are talking about, and it enables checklists for making sure everything is done.
• The structure of how each component breaks down into subcomponents. This identifies how the pieces of the system are grouped, which helps people navigate through the design to find the parts in which they are interested. (This also often reflects how the teams building the system are organized.)
• How each component interfaces and interacts with other components. This information provides a map to the other components that will be directly affected by this component’s behavior or structure, and vice versa. It provides the information needed to map out more indirect ways that other components can be affected or affect this component.
• The high-level state, behavior, and structure of each component. This is the core of the plan for how the component will implement its objectives. The state information might include modes of operation, information the component stores, or physical configurations. The behavior is how that state can change: how the component can change from one operation mode to another, or how the component can change its physical configuration. The structure might be the shape of a mechanical component, the modularization of a software component, or the major chip components of an electronic component.
• How each hazard or objective that has been identified for a component is addressed in the design.

All of this information should be annotated with a rationale for the decisions that led to the particular design.

37.1.3 Purposes of a design

Why should one take the deliberate and separate step of putting together a design for a system or component, rather than just implementing a component directly based on its specification?

For an exceptionally simple component, one can skip design and just implement the component—but the component must be truly simple, completely understandable from its implementation, involving no significant design choices, and with no future need to change the component, for this to pay off in the long run.

The value of an explicit design comes partly from its abstraction and simplification, and partly from being done mostly before putting together the detailed implementation.

• The effort of design provides time to reflect on decisions before committing to designs.
• A design effort allows incremental design that allows one to balance all of the important design constraints before going into detail on any one aspect of the design.
• The design provides a guide and explanation for the component.
• The design provides a place for people to write down the rationales behind the decisions they make.
• The design supports analysis, which usually feeds back into the design before spending resources on implementation.
• Designs for alternative approaches for implementing a component support comparison and decisions.

37.2 How designs are used

As noted above, a design enables communication among multiple people, across different times, and for different purposes.

37.2.1 Design leading into implementation

As well as all the uses listed above, the developer uses the design as a guide for the implementation. The resulting implementation must be consistent with the design: having the same structure and behavior, including all the functions in the design, and including no functions not in the design.

The developer or implementer must be able to understand the design to build a component that matches the design. The developer must also be able to check that they understand the design properly, so that there is a way to catch misunderstandings. A good design uses consistent structure, terminology, and diagrams to aid understanding. It provides a glossary of terms that may have multiple meanings to define how they are used in the design.

Developers will find problems with the design as they proceed through implementation. They may find ambiguities, where the design is unclear or where the design does not address some important condition. The developer may find errors, where the design is inconsistent internally or with its specification. The developer may find that parts of the design aren’t feasible to implement. All of these problems need to be fed back to designers for clarification or correction.

When the design changes, the developer needs to be able to identify what parts of the design have changed so they can change the corresponding implementation. The change might come in response to feedback from the developer, or evolution of the design to address changing needs or broader system fixes. This can be supported by using tools that track design versions and highlight design changes between versions.

Finally, the developer must be incentivized to follow the design (or provide correction feedback) as they implement the component. This includes having the designer and independent people review the implementation to compare it to the design. If they find that the design and implementation are not consistent, they must decide on how to change the design, the implementation, or both in order to achieve consistency. The component implementation should not be accepted as complete until they match.

37.3 Design artifacts

The artifacts that record the design enable all the usage cases listed above. The key functions they need to fill include:

• Organizing the design of the system and each component in a way that people can find the parts of they system they are looking for, and understand the context of those parts.
• Recording the structure, state, behavior, and shape of each component so that people can build a corresponding implementation or analyze the design to check whether it meets needed properties, including safety and security.
• Recording the rationale or explanations for why the design is what it is.

37.3.1 Supporting infrastructure

The designs for a system need to be available to everyone associated with the project, so that they can use the design to learn about the system and navigate through it.

An ideal solution provides a “single source of truth”: a user can go to one place and see all of the information about the system. The ideal solution also ensures that the user always sees a single consistent version of all the information. To the best of our knowledge, at present there are no systems that completely meet this ideal. However, there are ways to come close by integrating multiple tools and applying conventions to how they are used.

The infrastructure for maintaining designs needs to, at minimum:

• Store all of the design information, organized according to the names or identifiers for each part of the system
• Support multiple versions of the design, where one version is the current approved baseline, some versions are outdated and no longer useful except for historical reference, and some versions represent works in progress that are being developed or are alternatives, and that have not yet been completed and made part of the approved baseline. Note that software repository systems have evolved a number of useful conventions about how to maintain and name these versions; the version management capabilities for a design repository should be at least as capable as those.
• Support finding the differences between versions. When a design changes, a reviewer or developer will need to understand what they need to look at to understand the change. Tools that expose all differences between versions reduce the chances that some change will be overlooked.
• Support integration and tracing between specification and design artifacts. The design of a component is in response to the specification for that component, and the infrastructure should make it convenient for a user to see how the design maps to the specification and vice versa. Similarly, the specification of a subcomponent responds to the design of the component it is part of, and the infrastructure should help users understand that connection.
• Support reviews and approvals. Each design will begin as a work in progress, and then progress until it becomes the approved baseline (or until the work in progress is abandoned). Being able to track what designs need to be reviewed by whom, track the feedback from those reviews and corresponding updates, and tracking the approval of a design is error-prone without tools that support the process.

37.3.2 The artifacts

The following sections list the key artifacts that should be part of a design. Later chapters will detail these artifacts.

37.3.2.1 Breakdown structure

The breakdown structure consists of the hierarchical relationship of system, components, and their subcomponents recursively. It gives a name or identifier to each component, and provides the index or table of contents to the parts that make up the system. See ! Unknown link ref.

37.3.2.2 Control structure and other large-scale behaviors

A complex system will have behaviors or structures that cross multiple parts of the system, and don’t neatly fit within a single hierarchy of components. There are two important examples of these behaviors to document.

The first example is behavior or activity sequences that show how different parts interact with each other. These are sometimes documented as UML or SysML activity diagrams, which show how control or data pass among components, and how different components take actions in response to those. The point of these patterns is to show how components work together, which informs the interfaces, actions, and states that the components involved in the activity must support.

The second example is the hierarchies of control that operate in the system. These document how one part of the system controls the functions in other parts, including how some components provide sense data to drive the control logic, and how the control logic in turn sends commands to other components to effect control actions. Documenting and analyzing these control systems is an essential part of some safety and security process methodologies, such as STPA [Leveson11].

37.3.2.3 Details of each component

Each component in the system should have its own design. This is the primary content about individual components, as opposed to how components work together.

A component’s design can be represented in many different ways. However, it is easiest for users if all designs follow the same general format so that they know how to find particular kinds of information within every design.

All designs should include:

• An identifier and name or title for the component
• The version of the design and its status (in progress, obsolete, baselined)
• The position of that component in the breakdown structure (that is, what it is a subcomponent of)
• A short description of the purpose of the component, to aid people coming later to understand the design
• Information about the states, behaviors, interfaces, and environment for the component
• The decomposition of the component into subcomponents, if any
• A list of the artifacts that implement the component

Each component’s design should include rationale: the reasons why different design choices were made. This information helps those who must come along later to review or update the design.

In some cases, not all of this information can be represented in one way or in one tool. For example, for electronics designs the best way to represent some information will be in a CAD drawing that is maintained in a separate tool from the rest of the design information. In these cases, there should be unambiguous references from the main design to the CAD drawing and vice versa, and the versioning in the main design should be reflected in versioning in the CAD tool.

37.3.2.4 Safety, security, and other analyses

Part of the reason for developing a design—as a simplified model of what will be implemented—is to enable analysis of the design’s essentials. These analyses address whether the design will meet aspects of the component’s specification. These can include safety and security, as well as meeting business objectives, regulatory requirements, performance specifications, or resource budgets.

As I will discuss in the next section, it is recommended practice to develop these analyses incrementally in parallel with the design itself. In this way, a rough analysis of a rough design can provide quick, early feedback that will guide the design toward meeting its specified properties as it is developed in more detail.

These analyses become an important part of the record of a design once complete. They provide an extended rationale for why the design is the way it is. They may be needed to answer to external stakeholders, including regulators or courts of law, when it becomes necessary to provide evidence why the design is acceptable. The analyses also help people who must later evolve the designs to understand both the constraints on what they can change, and where they have freedom to make changes without invalidating the safety or other properties of the design.

37.4 Developing designs

As a matter of principle, the design for a system or component should be done after its objectives and specification are done, and before its implementation. Similarly, the design for the components in a system should proceed top down, starting with the system as a whole and proceeding to lower and lower level components. When the design of one component depends on the design of another, the two should be designed together.

These principles often lead people to conclude that systems should be built using a waterfall-like process, where everything is specified before design, everything designed before implementation, and so on.

Real projects are not so simple. I have never observed a project that actually used such a process, even when they tried to. This is because every complex system I have encountered is not fully and accurately knowable in advance. One can write a set of specifications that turn out to require some impossible component design. One might miss some important system objective when developing the initial system concept because the customer was not able to conceive of system operation until they could see part of the system in operation, or because the customer’s needs change. An initial design may be invalidated because a supplier discontinues an essential part. Some part of the system may require significant investigation or research before one can find a feasible way to approach its design.

All of these situations lead to cases where the specification, design, and implementation of the system does not proceed in a tidy one-way sequence through the waterfall stages. Instead, part of a component’s specification gets worked out, and some tentative design goes ahead using that part of the specification gets worked out. Or multiple possible design approaches are defined, and then someone proceeds to build simple prototype implementations of two or more of them to compare their feasibility. Or the design for a component must change, leading to a change in implementation. All of these may be happening in multiple parts of the system at once.

At the worst, all this change happening all over a system can lead to chaos where people working on different components are working to incompatible specifications or designs and building parts that will not integrate into a system. Project management may not be able to determine how much progress has actually been made on any part of the system, and thus be unable to detect when there are schedule or resource problems.

Therefore while the simple waterfall model, which organizes the work on a system, is not feasible, there is still a need to organize development work.

37.4.1 Applying general principles, flexibly

The principles I started with are good ideas in general, when used flexibly.

37.4.2 Additional principles

Chapter 38: Breakdown structure

The component breakdown, or breakdown structure, is the way to name and organize all the components that make up a system.

38.1 What is the component breakdown for?

The component breakdown organizes and names all the pieces in the system. It serves three main purposes:

• It provides a master list of all the components that need to be designed, built, and tested to make up the system
• It provides a unique identifier for each component, so that every piece of the system can be referred to
• It provides an organization for all the documents and artifacts that make up the system. In this role, it provides an index or table of contents that helps people find information about the system and its components

These purposes lead to a few objectives that a breakdown should meet.

• The breakdown should be complete. It should include all the major components that people will be working on.
• The breakdown should reflect how people naturally think about how components relate to each other. People should be able to navigate through the breakdown structure accurately to find information.
• The breakdown structure should result in identifiers that are easy to use. The identifiers will be written down in many different places; they will be embedded in different artifacts to link them together. Identifiers that are too long or too easy to mis-copy will cause difficulty for users.

38.1.1 Component breakdown versus work breakdown

Some institutions, notably NASA [NPR7120][NASA18] and other parts of the US Federal government [DOD22], specify the use of a work breakdown structure (WBS) in project management and systems engineering. A WBS as used in those projects is different from a component breakdown structure as defined here.

A WBS is oriented toward project management, not systems engineering. It is focused on defining the work to be done (hence the name) rather than the items or components being built by the work. From the NASA WBS Handbook [NASA18, p. 35]:

Other project management methodologies define a work breakdown structure as, in effect, a checklist of the kinds of work that may be required for a system, feature, or component. McConnell discusses using a generic work breakdown structure in estimation to ensure all the effort involved is accounted for [McConnell09, Table 10-3].

This difference in intent leads to two major differences in the contents of a WBS compared to a component breakdown. The first is that a WBS includes work items that are not product artifacts. The standard NASA WBS, for example, includes project management, systems engineering, and education and public outreach branches of the work breakdown tree [NASA18, p. 47]. Given that part of the goal of the WBS is to organize resources and budget for a project, that’s an appropriate choice. The other difference is that some people break a task for building a component down into multiple revisions or releases. For example, a “motor control software” component might have subitems “prototype”, “release 1”, and “release 2”, recording the phases of work done to develop that software package.

The component breakdown structure presented in this chapter is narrower in focus than a WBS. The component breakdown lists only the things that are being built. It must be complemented by other engineering and management artifacts to provide everything needed to run a project.

38.1.2 Component breakdown versus other views

The component breakdown is one of several views into the system’s design and specification. The component breakdown has only two purposes: listing all the components and giving them unique names, and providing a structure that people can use to navigate through the components to find one they are looking for.

The component breakdown is not for expressing other facts about components and relationships between them. There are other views and other breakdowns for representing that information—and for doing so in ways that are better suited to the specific information that needs to be explained. For example, a network or wiring diagram does a better job of illustrating how multiple hardware components are connected together. Mechanical drawings are a better way to show how components relate to each other physically. Data and control flow diagrams, perhaps realized as SysML activity and sequence diagrams, are better suited to expressing relationships between software components.

38.2 Basic concepts

When developing a component breakdown, the first question to be settled is: what is a component?

First, a component is something that people think of as a unit. Terms like “system”, “subsystem”, or “module” are all clues that people think of a thing as a unit. More generally, a component is something

• That can have a defined boundary of what functions or parts are part of the component, and what is outside, and
• That has a physical or virtual reality; it is something that needs to be built or acquired.

Components do not have to be atomic units. Systems have subsystems; components have subcomponents. For example, the electrical power system (EPS) in a spacecraft is a medium-level component in a typical breakdown structure. It is part of the spacecraft as a whole. It is made up of several subcomponents: power generation, power storage, power distribution, and power system control. Each of those subcomponents in turn have constituent components themselves: for example, power generation has solar cells, perhaps arrays that hold the cells, perhaps some other power generation mechanism.

This illustrates the general pattern for the breakdown structure. The structure is a tree, with the highest-level component being the system as a whole. The system as a whole is typically not just a vehicle or box; it is the entire mission or business on which a vehicle is part. Underneath the whole system come the major component systems. For a spacecraft mission, this might be the spacecraft, ground systems, launch systems, and related assembly and test systems. The next level of components are the major subsystems. The structure continues recursively until reaching components that are the smallest that are sensible to model using systems tools.

The recursive process of defining smaller and smaller components ends when there is a judgment that further subdivision won’t help the systems engineering process. In practice, for example, continuing the breakdown structure all the way to individual resistors and capacitors on a printed circuit board is too detailed to be useful for systems engineering tasks.

Some criteria I have used for deciding when to continue subdividing a component into subcomponents include:

• Does the state of the component separate into multiple independently-acting parts, or not? If the state of the component can easily be modeled as a single state machine, then it probably does not need to be refined; if the natural modeling is as a collection of independent, interacting state machines, then it should be refined.
• Is the component physically a single physical unit? Or is it fundamentally a collection of smaller physical components?
• For software, is the component treated as a single unit? Or does this project assemble the software component out of other parts?
• How large or complex are the tasks to design, build, or test the component? If the component can be designed and built by one person or one vendor, then there is less incentive to split it into subcomponents than if multiple people or organizations must work together to design and build the component.

Some examples:

• An aircraft. This is a high-level component in a breakdown structure for a project that is about designing and building a new aircraft model. In that project, the aircraft will have components like body, wings, propulsion, fuel, communication, and so on. If, on the other hand, the project is to design and build an air traffic control system, each aircraft is likely best treated as a low-level component that is not further broken down because all the aircraft in the airspace have roughly similar functions and behaviors, and the air traffic control system is concerned primarily with sensing those aircraft and guiding their flight.
• An aircraft’s body (fuselage) structure. This is usually a middle-level component in an aircraft system. It is part of the aircraft over all. It is made up of formers, stringers, trusses, and skin. Each one of those subcomponents is typically built separately and then joined together to make the aircraft’s structure. Each will have different mechanical constraints; there are likely to be different choices for the materials out of which each one is built. These are candidates for refinement of the fuselage as a whole into a collection of subcomponents.
• Infrastructure software, like an operating system. An operating system is a complex body of software; some operating systems contain several million lines of code. If a project is building an operating system, then the operating system as a whole will clearly need to be broken down into many subsystems. However, for a project that uses the operating system but doesn’t build it, the operating system doesn’t necessarily need to be broken down further: the team will obtain the operating system as a whole from a vendor and use it in its entirety without the team needing to make modifications to the operating system itself. For an aircraft project, for example, the team may buy a real-time operating system from a vendor. The team may need to build or separately obtain some of the related software components, such as a board support package and device drivers. In this case, the OS would be one component, and the BSP and drivers would be separate subcomponents–but there is no need to model the operating system’s virtual memory management system.
• A printed circuit board implementing a processing unit. This is an example that may or may not be worth breaking down into subcomponents. If the board (and the parts mounted on it) are treated as a single entity, acquired as a whole, and treated as a single unit for application software purposes, then the board need not be broken down further. On the other hand, many circuit boards implementing a processing unit today are complex systems: in one recent example, the board had a primary processor, a secondary control processor, a storage system, a cryptographic unit, and two different network communication units. While the board was built and delivered by a single vendor, separate software had to be written for each of those parts—and in some cases there was an option whether to populate some of the storage and communication parts or not. In that example, breaking the overall board into subcomponents was necessary.

38.2.1 Satisfying the objectives

1. Completeness. Completeness depends on the exercise of identifying all the components to run to completion. The hierarchical approach does not directly inhibit or support this objective. However, the hierarchical approach makes it easier to approach completeness iteratively: one can start with a high-level breakdown, and incrementally expand parts of the breakdown tree when one finds that some components need to be refined.
2. Supporting navigation. People generally talk about the structure of systems in a hierarchical way: system and subsystems and components and subcomponents and so on. This means that a hierarchical breakdown structure matches common usage (as long as the refinement into smaller and smaller components follows the common usage).
3. Usable identifiers. The hierarchical structure does support unique names for each component, as will be discussed later. The identifiers are usually not the most compact possible, because the identifiers reflect a path of names through the breakdown structure tree, similar to the way file systems and URLs organize hierarchical names. However, the hierarchical identifiers in practice have worked well as a readable and writable form of identifiers in other domains, including URLs.

38.2.2 Alternatives

The approach laid out here is fundamentally hierarchical, and reflects the way people usually approach breaking down a complex system—by a reductive approach that organizes parts into a hierarchy.

That is not the only approach to organizing the components. Mechanical and electrical engineering systems often use a more-or-less flat space of part numbers to identify components. The specifications for each part can have attributes, and the attributes allow one to search for a desired part.

A flat part number approach works well for low-level, physical components. A 100 ohm resistor can be used in many different components; there is little value in giving a different name for its use in one place on one board and a different name for a second place on that board, or on a different board. Similarly, when manufacturing many instances of a vehicle, using a part number to identify the part in an assembly works well.

I have generally not used a part number approach for higher-level systems activities, however, because the uses are not the same. During design, each component that systems engineering deals with is generally unique.

38.3 Component identifiers

A component’s identifier provides a unique way to refer to that component. It is like the address for a building: it allows one to find the component (or its specifications), but does not by itself convey much more information. The keys are that the identifier be unique, and that people can use the identifier to find what they are looking for.

The pathname is the long-standing practice for creating identifiers for elements in a hierarchy. This is familiar from file systems and URLs: the path /a/b/c/d refers to a file or object named “d”, which is contained in “c”, which is in turn contained in “b”, which is part of “a”, which is one of the top-level objects or folders in the system. While the object name “d” is not necessarily unique (there can be another object /a/f/d, for example), the path as a whole does give a unique identifier for the object or file.

This approach applies to the identifiers for components in a breakdown structure as well. The names in the path are typically separated by a slash (/) or period (.).

The names of each component in the tree can be abbreviations or short words describing the component. Both work well; the choice is primarily a matter of style. When there are commonly used abbreviations for some components, it is reasonable mix and match abbreviations and longer names. For example, a spacecraft’s computing system is often called the CDH (command and data handling); attitude control is the ACS (attitude control system); and the electrical system is the EPS (electrical power system).

Some examples from a fictitious spacecraft system:

Long component identifiers can become a problem. Long identifiers are harder to type than shorter ones. Sometimes there are limits on how long an identifier can be; for example, if one is recording information about components in a spreadsheet and putting each different component on a different sheet, most spreadsheet packages have limit on how long a sheet name can be.

The length of an identifier is driven by how deeply the breakdown structure tree goes. The path name for a component six layers down in the hierarchy will be much longer than the path name for a component in the third layer. This suggests that one should try not to make the component hierarchy any deeper than it needs to be.

38.4 Viewing the breakdown structure

Many people find a visual representation of the breakdown structure helpful for understanding it. Here is a drawing of an incomplete breakdown structure for a simple spacecraft:

It is worth finding tools that can show this kind of visual representation of the breakdown structure.

38.5 Context and relationships

The breakdown structure provides the fundamental organization for most systems engineering artifacts. This means that the structure chosen for the breakdown will affect how most other parts of a specification are organized.

Each component named in the breakdown has a specification. The specification includes information like

• The purpose and scope of the component
• Its requirements
• Its state and behaviors
• Its interactions and interfaces with other components

When two components interact, the interface between them must name which components are involved. The specifications for each component must indicate what data or control they will be sending and receiving in the interaction.

The identifier for a component provides a way to express a reference between implementation and test artifacts, like source code or drawings, and the specifications to which they should comply.

The breakdown structure affects almost everyone working on the project. This includes:

• Systems engineers develop the breakdown structure
• Every engineer and developer users the breakdown structure to find information about the components they will work on or that they will interact with
• Project managers use the component breakdown as a basis for defining development tasks and tracking their progress
• Contract managers use the breakdown structure to identify deliverables that vendors might provide

38.6 Advice

38.6.1 Evolution

The understanding of the system evolves gradually from the initial concept to the time that a final product is delivered (if indeed there is a final product). At each step of this evolution, the understanding of what should be in the breakdown structure and how it should be organized will change.

Because the breakdown structure is central to many other processes and artifacts, a change to the breakdown structure will result in changes to potentially many other artifacts. The cost of the change grows as the size of the breakdown structure tree grows.

Don’t try to build an elaborate and complete breakdown structure too early. At the beginning, while still working out the basic concepts of the system and its structure, just sketch out the first level of the structure—and try out several potential structures until one appears to match the system’s objectives. Often the main structure will be suggested by common practice for similar projects: the automobile industry has a common, vernacular breakdown of cars and trucks into common subsystems, for example.

In general, it is best to keep a branch of the breakdown structure shallow as long as there is significant uncertainty about how that part of the system will be designed. In an aircraft, for example, the propulsion system should be left unrefined in the breakdown structure until the team has settled on the general approach to propulsion—will it use turbofans, turboprops, propfans, electric rotors, or some combination? The broad choice can typically be settled early in concept development by working out the concept of operations and determining what capabilities, performance, and physical layout will meet the aircraft’s operational needs. Once the general architecture has been decided, then one can refine the propulsion system by adding a layer of components for each engine or other major unit involved in propulsion.

38.6.2 Depth

The point of the breakdown structure is to help people find and refer to components. The breakdown structure should reflect common ideas of how a system breaks down into components, and should result in short, easy-to-use identifiers. The breakdown structure should focus on these capabilities and not be drafted into serving other purposes.

Consider the breakdown structure for all the sensors that provide information to an autonomous vehicle. One way to organize the sensors is to create a general “sensors” component, and then include all the sensors as children of the general sensors component. Another way is to break the sensors down first by general type (camera, lidar, radar, sonar, microphone), then by general location of the sensor on the vehicle (front, left, right, top, back), and then by the specific sensor unit. In this example, the first approach leads to a shallow and broad breakdown structure; the latter example leads to a narrow and deep structure.

In general, a shallow, broad breakdown structure will meet these objectives better than a narrow and deep structure. There are a few reasons for this.

• The identifiers for sensors will be shorter. The shallow structure might name a particular camera “sensors.leftcam” while the deeper structure might name it “sensors.camera.left.camera1”.
• The broader structure serves the purpose of helping people locate a component better. In the broad structure, a developer knows that all the sensors are under the “sensor” component; they can navigate to the sensor component and then find the particular sensor they are looking for in the list of sensors. In the deep structure, the developer must navigate to sensors, then to cameras, then to the location, and then find the sensor. If the developer is looking for a particular type of camera and doesn’t know where cameras of that type are located, the extra layers pose extra work.
• The shallower structure handles evolutionary change more easily. If a project uses the deeper approach, then determines that the layering structure isn’t working as well as expected and the structure needs to change, that will lead to changing the identifiers for many components—which in turn leads to finding and updating identifiers in many other artifacts where the identifiers have been written down. In the shallow structure, there is no structure organizing the sensors and nothing to rearrange.

This leads to a general principle. The breakdown structure should be used only for providing a unique name, and not for embedding a taxonomy or search attributes. The tools that people use to navigate through the breakdown structure and its related artifacts, like specifications, should provide search mechanisms that let someone find a component by attributes. Embedding extraneous information, like a location attribute or model number or power requirement in the name will just make the names longer, harder to use, and less resilient to change.

38.6.3 Multiple fit

The hierarchical, tree-structured approach recommended here makes each component part of exactly one parent component. It does not accommodate components that have more than one natural affinity to parent groupings.

Consider a radio transceiver that is used to communicate between aircraft, such as the ADS-B systems used for collision avoidance. This transceiver could be categorized multiple ways. It is part of the aircraft, but it is also part of an air traffic management safety system. The transceiver within the aircraft is part of a communication system, but it is also a part of the flight control system and intimately connected with human interface components on the flight deck. The transceiver, in other words, is part of several different groupings of components, depending on who is looking and for what purpose.

There is a fundamental tension between simple organizing structures, like a tree, and the richer relationships that elements of a system have with each other. For an excellent discussion of this, see Alexander’s essay on trees as a structuring approach for cities [Alexander15]. In that essay, Alexander proposes that a lattice structure is a more appropriate model for organizing urban structures. In his account, a tree-oriented description of a city fails to account for the ways that a house can be both a place for a family to live as well as a node in a social network and a place of work; in each of these roles, the house is related to different buildings or locations in the city.

The systems engineering approach presented here addresses this problem by separating naming or identity from the complex relationships that each component actually has. The breakdown structure only tries to give a name to each thing, like the address for a building. The relationships, functions, requirements, and everything else that goes into defining a component are all left to other artifacts, such as the component’s specification and models of the components.

This means: don’t try to make the breakdown structure do too much. When a component fits into multiple categories, pick the one that seems most natural for most users and leave it at that. Other artifacts and tools will address greater complexity.

38.6.4 Not by function

The breakdown structure is for organizing components: things that are built and that can be seen or touched (possibly virtually).

There is sometimes a temptation to try to organize system functions into the breakdown hierarchy. Don’t do that. The breakdown of function—and of the allocation of function to component—is a separate task that needs to be addressed by a structure that focuses on how functions are organized.

A better approach is to maintain the component breakdown and a functional breakdown separately, and maintain an allocation mapping that shows how different subfunctions are achieved by different components. The functional breakdown is often better reflected in the structure of how specifications or requirements derive from each other. See the chapter on requirements for more on this.

38.6.5 Keep related things together

Some projects have proposed organizing components primarily by some fundamental, nonfunctional attribute. One project was considering separating hardware from electronics from software from operational procedures at the top level, and then organizing components within each of those categories by subsystem. Another project organized components first by the vendor organization that was to implement the component.

These approaches make it harder for people to use the breakdown structure to find things. Consider an electrical power controller on a spacecraft. This has an electronic component (the board and processor that runs the power control function) and a software component (that makes the decisions about what to power on and off, and to report information to a telemetry function). Someone working on the power controller will generally want to know about both aspects. Requiring them to look in two widely-separated parts of the breakdown structure is inconvenient, and (more seriously) it increases the chances that someone will miss a component that they need to know about to do their work.

As a general principle, it is better to group components by how people naturally think of them as being grouped. Keep functionally-related components close together in the breakdown structure so that people find everything they need about something by looking in one place.

As noted above, this doesn’t always work. The breakdown structure will not be perfect because not everything in a system naturally falls into a hierarchical organization. But the more that like things can be grouped, the easier it will be for people.

38.6.6 Generic and reusable components

There is one special case of a component fitting into multiple places in a breakdown structure that deserves special treatment: generic and reusable components.

Consider an operating system. There may be multiple processors within a system that may all run instances of the same operating system. It is useful to have one specification for that operating system: there’s one product that is acquired from a vendor, there is one master copy kept somewhere, and so on. At the same time, that operating system will be loaded onto many different processor components in different subsystems.

One way to address this is to have a part of the breakdown structure for generic components, and then put an instance of that component in the places where it is used. The specification of each instance component can refer to the specification for the generic, with those functions or requirements that are specific to the instance added. This is an example of using the class-instance model from object-oriented programming to solve the problem.

38.7 Examples

38.7.1 NASA Work Breakdown Structure

The NASA project management process and systems engineering standards use a common WBS structure across all NASA projects. The use of the WBS is codified in a Procedural Requirement document [NPR7120], with details in an accompanying handbook [NASA18].

The NASA WBS is used as a project management artifact to organize work tasks, resources and budget, and report progress. The hierarchy must “support cost and schedule allocation down to a work package level” [NPR7120, p. 113]. A “work package” means one task or work assignment that is tracked, budgeted, and assigned as a single unit.

A NASA project’s WBS tree is rooted in the official NASA project project authorization, with its associated project code.

The first level of elements is defined by NASA standards, and each element has a standard numbering. The standard elements for a space flight project are: [NPR7120, Fig. H-2, p. 113]:

Note how this organization mixes technical artifacts (payloads, spacecraft, ground systems) and management activities (project management, safety and mission assurance, public outreach).

The NASA WBS is intended to be one part of an overall project plan document. The project plan also contains information like:

• The governance and authority for the project
• Resourcing
• Organization (team) structure
• Schedule and cost plans
• Safety and risk plans

38.7.2 MIL-STD-881 Work Breakdown Structure

This breakdown structure standard aims to provide a “consistent and visible framework” [DOD22] for communicating and contracting between a government program manager and contractors that perform the work. It addresses needs such as “performance, cost, schedule, risk, budget, and contractual” issues [DOD22, p. 1]. This kind of WBS is thus focused on supporting contractual relationships with suppliers.

The standard defines a number of different templates for different kinds of projects. It includes templates for aircraft systems, space systems, unmanned maritime systems, missiles, and several others.

The template for an aircraft system includes the following Level 2 items:

• System integration, assembly, test, and checkout
• Air vehicle
• Payload/mission system
• Ground/host segment
• Aircraft system software release 1..n
• Systems engineering
• Program management
• System test and evaluation
• Training
• Data
• Peculiar support equipment (support equipment specific to this air vehicle)
• Common support equipment
• Operational/site activity by site
• Contractor logistics support
• Industrial facilities
• Initial spares and repair parts

As should be clear from this example, this WBS template aims to address not just the design and building of a system but rather the operation of the entire program, including testing, deployment, and initial operation.

38.7.3 A simple spacecraft system

This is an example component breakdown for a simplified imaging spacecraft. The spacecraft uses solar panels to collect energy; it has a single imaging camera to collect mission data; it has a flight computer to run the system; an attitude control system to point the imager where needed; and a radio to communicate to ground. (The graphical version of this breakdown structure is included earlier in this chapter.)

This example only goes four levels deep. The actual breakdown structure would likely include at least two more levels, to represent, for example, different parts of the flight control software or subcomponents of the radio transceiver.

The example includes an example of a component that could fit in multiple places in the structure: the propellant tank heater. This is part of the thermal management system—its function is to keep the fuel in the propellant tank within a certain temperature range—but it is also part of the propulsion system. In this example the choice was to categorize it as part of the thermal management system.

Chapter 39: Component design

• what goes into a component’s design, based on Section 11.4

39.1 Form

39.2 State

39.3 Actions or behaviors

39.4 Interfaces

39.5 Non-functional properties

39.6 Environment

Chapter 40: Good design practices

• Modularity
  • Isolating high-interaction parts

• Hierarchy and composability

• Avoiding long distance

Chapter 41: Prototyping

- purpose of prototyping

- reasons to prototype

- pitfalls of prototyping

- good practices

– bounded effort

– explicit non-reusability

Chapter 42: Budgets

• Idea of budgets: that there is some amount of resource to be shared among a number of components (parts of the system)

• the problem to solve: use resource efficiently but not oversubscribed (not too heavy or too power hungry)

• Examples: simple
  • Power, constant source
  • Mass
  • Bandwidth or time

• Examples: complex
  • Power with variable generation (show powerstrip output)
  • Screen real estate, especially when modal
  • Fuel/Delta-v

• Techniques
  • Tracking
  • Tracking with modality or variability
  • Interaction with specifications: system-level requirement not to oversubscribe, but not on subcomponents. can say “mass must be under X” but that doesn’t work as expected
    • The difficulty: give everyone a target and they will work to that target but not past it; when some component can’t go down to target the system ends up oversubscribed
    • the example about asymmetric incentives
  • Estimation — initial and improving
  • Reserves that decrease over time

• Need: give incentives for teams to go under a target or use a shared target
  • Leadership effect
  • Marketplace mechanism?

Chapter 43: Safety and security design

• Introduction
  • what this design aspect is about
  • define safety, security, and other similar properties
  • how this weaves throughout the development phase

• Objectives
  • design and build a system that meets customer or regulatory goals for keeping things safe, secure, or similar properties
  • gather objective evidence that the system meets those goals
  • support fixing problems when they are discovered
  • support evolving the system without breaking these properties

• Key terms
  • Incident
  • Hazard
  • Accident
  • Risk

• General method
  • picture of general flow
  • define the objectives

• Safety-specific design
  • relation to safety standards

• Security-specific design
  • working out potential threat actors

Part IX: Implementation

Chapter 44: Introduction

• What the implementation phase is about
• References to the vast body of literature
• Specific techniques discussed here
• Managing technical debt, unknowns, and risk

Chapter 45: Good implementation practices

Chapter 46: Integration-first implementation

• Definition
• Purpose for managing integration risk early
• Dependencies among components and implementing in parallel or sequentially
• Examples
• Relation to plan
• Relation to methods (next)

Chapter 47: Skeletons and mockups

• Purpose of tackling integration uncertainty and risk first
• Definition
• Relation to prototypes

Chapter 48: Implementation using testing

• Purpose of using testing during implementation
• Comparison with TDD
• Tools

Part X: Verification

Chapter 49: Introduction

• purpose
• verification implementation
• verification methods
  • when to use each
• inspection and review
• inspection implementation
• analysis
• analysis implementation
• testing
• testing implementation

Chapter 50: Recording evidence

• traces from purpose to specification to design to implementation
• records of testing
  • what the test was, and what it verifies
  • linked to specification and implementation artifacts
  • testing procedure used
  • record of the execution
  • pass/fail result, and log of problems
• records of analysis
  • what it was to verify
  • methodology
  • output of analysis
  • pass/fail result, and log of problems
• historical evidence from earlier versions
  • any evidence is specific to a version of purpose, specification, and implementation (or design); invalidated after any of those change

Chapter 51: Good verification practices

Bibliography