Making systems—2: Life cycles
VI   Reference life cycle
Chapter 31   Using the reference life cycle

The last several chapters have presented a reference life cycle pattern. This pattern is intended to inspire thoughts about how a project can organize its own work. It is not in itself ready to use off the shelf. Each project will have its own needs, its people will have preferred ways to work, and some projects will have life cycles mandated by regulation or industry standards to follow.

undisplayed image
Figure 31.1: The entire reference life cycle.

XXX purpose of the life cycle – get a system built, deployed, and sustained that meets customer and other stakeholder needs – in doing so, develops the whole system including all its development artifacts, not just the end product – plea for thinking about flexibility with discipline, and the idea of the artifact edifice

The reference pattern does not discuss the roles involved. A full definition of each phase will include definitions of who performs different tasks in each phase, and in particular who is responsible for milestones. I argue elsewhere (e.g. Section 8.2.6) that, to be meaningful, reviews must be done by people with an independent perspective on the material being reviewed, and that approvals must reflect a check on the work fitting into the project’s big picture.

The objective for any project is to develop and adopt a life cycle that meets its needs. In the next section I discuss several principles that a good life cycle will follow; these can help people evaluate a life cycle they are considering. Some other considerations include:

The project works out what life cycle patterns it uses, and documents the patterns. This effort starts during project preparation (Chapter 26). It does not necessarily need to define the entire life cycle all at once; it can be done iteratively, as long as the work keeps ahead of what the team needs. In practice I have found that enough should be completed in the project preparation phase that the team understands the general complexity of the work ahead, has chosen a development methodology, and can name major milestones they will need to meet. The remainder of the life cycle can be worked out during purpose and concept development and likely will be refined or adjusted as the project moves along. (I worked on one project that had a limited budget, and spent much of that budget writing elaborate management and engineering plan documents before even beginning to work out the high-level system concept. The result was a pile of such documents that were never looked at again, which was a waste of their efforts.)

In no case should the team get ahead of the defined life cycle. See Section 8.1.5—Principle: Team habits for a discussion of this principle.

The life cycle patterns have value only if the team actually uses them. This means that the team must know that the patterns exist, understand them, and agree that they are useful. The people in the team must also understand that they have a responsibility to follow the patterns, or to raise an issue when they find a problem with the life cycle’s definition. Achieving these means educating people as they join the team about what the life cycle is and how to learn about it, as well as monitoring that everyone actually follows the patterns. The team can also learn about and accept a life cycle a bit more easily if they are involved in developing the patterns; at minimum, they should be able to give feedback before the patterns are adopted.

The life cycle patterns are documented in a way that the team can find them and learn about them when they are joining the team and when they need to refresh their understanding of how some step works. The documentation is an artifact that should be managed using the principles in Section 17.4: it should be versioned and under change management; it should be stored in a way that the team can find it when needed; and it should be secure enough that it will not be tampered with.

There is no one right way to document them, as long as the documentation is well-organized and accessible. Some organizations prefer to define the life cycle in a prose plan document, which can be printed in its entirety if needed. I have had some success maintaining the documentation in a wiki or in a collection of web documents; the advantage of these is that they allow linking between parts of the document. The patterns should be explained and listed explicitly; they should not be hidden in a workflow system that doesn’t let team members see and understand the whole context for their work (see Section 4.6 for an example).

The documentation for each phase or step in the life cycle should include the information listed in Sections 23.5, 23.6, and 23.7.

31.1 Meeting life cycle principles

In Section 23.10, I listed principles that a life cycle pattern should meet. The reference life cycle pattern in this part reflects these principles, though it cannot address all of them. Here are ways that a life cycle built using this reference as a base can address them.

Know the purpose for something before developing it. The development phases in the reference life cycle all start with a purpose development step, in which the purpose for the system, component, or feature gets worked out before proceeding on to concept and design. The system evolution phase reiterates these patterns.

The project preparation phase is a time to think about the purpose for the project as a whole, and to work out the purposes for the different aspect of project operations; for example, what the team organization should achieve, or what is expected of life cycle and procedures.

Documenting these purposes means that when the questions are revisited—and they will be—people can understand the reasons why decisions were made, instead of forgetting why and making up new and probably different reasons.

A good life cycle definition will ensure that these phases have review and approval milestones that check that the purpose has been worked out and documented.

Build in time for and incentivize deliberative thinking. The concept steps in development and evolution support this kind of deliberation, as long as the team culture actually incentivizes taking the time to work through a concept deliberately.

The procedures and instructions for reviews complement the life cycle patterns by prompting reviewers to ask questions about deliberations taken, and encouraging them to reject work that has not been thought through. Again, projects that are in a rush will tend to disincentivize this, usually storing up trouble for themselves for later. The project leadership can create an example and incentivize taking enough time to think.

Assign decision-making authority to an appropriate level based on the nature of the decision. The reference life cycle does not address this as written. The structure of the team, and how roles are organized in the team, complement the life cycle and determine how authority is distributed. The specific decisions about what role can take what decisions is encoded in the details for life cycle phases and in the procedures that apply during those phases.

For more details, see ! unknown reference XXX.

Build in ways to check work, and design them so they are a team norm and not prone to triggering defensive reactions. The reference life cycle includes reviews at regular points in the work in order to support this principle. The definitions of procedures for reviews augment the life cycle by making it clear what is to be reviewed and how people are to go about the reviews.

Build for the longer term. The reference life cycle supports this somewhat by providing development steps when thinking about how to design for the long term and when documentation to support future revision can happen. A project can define more specifically what kinds of documentation is expected from development phases, and review procedures can make it clear that such documentation must be provided before a piece of the work advances in its development steps.

Project-wide decision points. I have pointed out some times when the life cycle might have review and decision points, such as after purpose development, during concept development, and in the acceptance phase at the end of development.

Think about exceptions that might happen, how to handle them, and when to change course. I have not tried to address this principle in the reference life cycle. Working out how to handle exceptions is a process like designing for safety or reliability (Chapter 46): working out the kinds of hazards (exceptions) that might be foreseen, then deciding what should be done about each one. The particular kinds of exceptions depend on the project: a delay in getting a new funding round affects a project in a startup but not a small project in a well-funded organization, for example.

Some kinds of exceptional conditions are not really a matter for the life cycle, but rather for the development methodology, procedures, and planning approach that the life cycle patterns organize. Risk management (! unknown reference XXX) and the way that planning accounts for uncertainty (! unknown reference XXX) are ways to anticipate specific exceptional conditions and, in many cases, avoid them.

The choice of development methodology affects how easily the project can adjust when it needs to change direction (Section 28.5).

Define the work so that everyone on the team can agree when a step has been completed. This is achieved by clearly documenting each step or phase in the life cycle.

Give a clear definition for each step of the quality considerations by which the work can be judged. Similar to the previous principle, this is met by the documentation for each phase or step.

Make the pattern as light-weight as possible without compromising quality. The reference life cycle in these chapters is only a skeleton of a complete life cycle definition. I believe that everything in it is necessary for most projects, though some projects will likely be able to trim out some parts. As long as a project’s life cycle does not add too much to this reference, the life cycle itself will likely be acceptably lightweight.

There are three ways that I have seen a project end up with a too-heavyweight process. One is to add too many new phases or steps to the life cycle, to the point that people in the team have trouble figuring out where the project is and what steps they should be doing. Another is to make the work inside one phase too complex: adding more reviews than the minimum necessary, for example. The third is when the procedures that say how to do parts of the steps get complex. In Section 4.6, I discussed how complex one procedure (in this case, for qualifying component vendors) caused problems for a large launch vehicle project.

31.2 Relation to NASA life cycle

As many people are familiar with the NASA life cycle, or may be obliged to use it (or a variant of it), in this section I discuss how the canonical NASA life cycle compares to this reference life cycle. I will use the general NASA life cycle defined in NPR 7120.5 [NPR7120, Figure 2-5]. I presented an overview of this life cycle in Section 24.2.1.

The NASA life cycle is divided into seven major phases:

  1. Pre-Phase A: concept studies that define a potential mission.
  2. Phase A: concept development that results in the definition of a feasible and useful mission.
  3. Phase B: high-level design of the system for a mission and evaluation of the technology available for the mission.
  4. Phase C: the bulk of development, in which the system components are developed, verified, and manufactured.
  5. Phase D: assembly, integration, and test of the flight and ground systems; launch and initial on-orbit checkout.
  6. Phase E: operation in flight.
  7. Phase F: close out, including flight system disposal.

This life cycle was developed over several decades as NASA learned how to develop and operate complex missions. Elements of this approach have been adopted by many other organizations—terms like “System Requirements Review” and “Preliminary Design Review” have become nearly ubiquitous in the aerospace industry.

The overall flow of the NASA life cycle is organized around two constraints: fitting in with the US Federal funding cycle, and managing risk for a few highly expensive steps. The funding constraints come at the transition from Pre-phase A to Phase A, when the mission is approved and funded enough to develop its concept, and between Phases B and C when the agency commits to funding the full mission [NASA16, Section 3.5, p. 25]. The distinction between Phases C and D comes with Phase C covering development of designs and fabricating components, but actual assembly of a spacecraft does not start until Phase D, at which point there should be little residual risk that the system design will not work out.

undisplayed image
Figure 31.2: NASA life cycle mapped to reference life cycle

The NASA approach was, however, developed for hardware-heavy systems and people who today develop spacecraft or aircraft that have a greater amount of software components sometimes find it difficult to map software project best practices onto the NASA approach. There are usually two issues: software development best practice puts integration earlier than the way many people interpret the NASA model; and many software developers combine design and implementation, especially for novel software functions. I show one way to reconcile these approaches in the mapping in this section.

The reference life cycle I have presented is organized around types of work—conception, specification, design, and so on. The NASA life cycle is organized at the highest level around milestones that check progress early, allowing corrections before committing agency resources. This means that the NASA life cycle splits several of the early phases in the reference life cycle in two, with a major review or checkpoint of the project’s progress before continuing. These two approaches are compatible: almost every project will have some kind of project-wide milestones alongside the milestones specific to the work phases.

In the following, I present how each of the NASA phases maps to the reference life cycle.

31.2.1 Outside the NASA life cycle

The reference life cycle defines the project preparation phase and project support “phase”. The preparation phase involves a rough definition of the project and establishing basic operations abilities. Project support covers support functions, like managing teams, finances, or artifacts.

In the NASA environment, the initial support is provided by one or more agency centers and external collaborators, using budget, tools, space, and people for general concept exploration. Each center has its own procedures for starting up a concept exploration project.

Similarly, the NASA agency provides essential support services to its projects.

In one project I worked on, the NASA Ames Research Center had a Mission Design Center that was charged with exploring potential mission concepts. A small group developed the mission idea and explored ways it could be realized. Ames and the agency provided all the key support infrastructure: staffing, finance, office and lab space, and IT services, for example.

31.2.2 Pre-phase A—Concept studies

The Pre-phase A work develops a concept for a mission, presumably in response to NASA agency priorities. It is expected to limit its work to the concept of a mission: what it might achieve, who would benefit from the mission, and high-level technical approaches that might support such a mission.

There is one major review in this phase: the Mission Concept Review (MCR). This checks that the potential mission is well formulated and that there is sufficient interest to justify funding “project formulation”—working out a detailed concept and high-level design.

At the end of Pre-phase A, after the MCR, the agency makes a decision whether to continue the project and fund it for “formulation”: the phases where the concept and high-level designs are worked out. This involves greater financial commitment than the early studies, and is the start of the “real” mission.

Pre-phase A maps to the purpose development phase (Section 28.3) and part of the concept development phase (Section 28.4). The purpose development phase covers identifying what the mission might do, and who the mission stakeholders might be. The concept development produces an initial sketch of a mission concept, without breaking the concept down into great detail.

31.2.3 Phase A—Concept and technical development

This phase is the first of two that are about developing a feasible high level design for a mission and ensuring that necessary technologies are available. Phase A includes developing a complete mission concept and high-level system designs. The team identifies any new technology that the mission will require and works out what will be needed for it to be ready to use in flight.

The depth of design and requirements is not clearly specified in the NASA procedural documents. However, my experience is that it is generally taken to include the spacecraft and its major subsystems, ground systems and their major subsystems, potential launch vehicles, and testing and other ground support equipment to a similar level. The exercise is intended in part to develop the general structure of the system and its likely cost, and in part to find those parts of the system that will require new technology.

Phase A includes developing a list of new technology that will be used for the mission, an evaluation of its maturity, and plans to develop that technology so that it will be mature enough for flight.

This is the first phase where a NASA project is funded for itself, as opposed to using resources allocated for general mission concept development. The various management and development plans required by NASA procedures get developed in this phase.

Phase A includes two key reviews:

The NASA Phase A maps to the second part of the concept development phase in the reference life cycle, along with concept and specification and preliminary design steps for the highest-level components in the system.

31.2.4 Phase B—Preliminary design and technology completion

Phase B continues the work from Phase A, completing a preliminary design and refining any new technology to the point where it is sufficiently mature to use in flight. This often involves building models and prototypes of parts of the system.

Phase B also involves developing the safety and security of the mission. The high-level design should incorporate designs for safety, security, and other critical mission success factors, and the design should be backed up by analysis showing why the design is sufficient. (See Chapter 46 for more on safety design.)

At the end of Phase B, the project should have a high-level design for the entire mission. That design should meet all the mission objectives, be technically feasible, and fit within cost and schedule available.

After Phase B, the agency allocates money to actually implement the system. The process can be complex and time-consuming, potentially involving legislative approval. The estimates for cost and schedule should be accurate enough that the project is unlikely to exceed them, which would require repeating the process to find more funding or time. This imposes limits on how much risk the project can carry going from Phase B to Phase C.

There is one key review in Phase B:

The end of Phase B maps to a slice across the development phase in the reference life cycle. It includes the concept, specification, and preliminary design of the first two or three levels of components in the breakdown hierarchy (Section 11.3; Chapter 41). In general this might include the major spacecraft subsystems—payload, structure, propulsion, attitude control, and so on. The portion of the design step includes prototyping or modeling of components that pose technical risk, and the design may go to deeper levels of the breakdown hierarchy if needed to understand and address that risk.

The mission-level PDR follows reviews of the component-level preliminary designs.

31.2.5 Phase C—Final design and fabrication

This phase is when most of the development and production work is done. It involves designing, building, and verifying all the components in the system, to the point where they are ready to be assembled into the working spacecraft and ground systems.

Phase C is designed around the spacecraft being difficult and expensive to assemble, involving building large structures, using complex manufacturing tools, threading complex wiring harnesses through the structure, and putting large amounts of money at risk during the assembly. This leads to organizing the final assembly work to avoid as much risk as possible by ensuring that all the components are ready to assemble before committing to the final assembly steps.

During this phase, the team completes all of the designs and implementations of the system components, and verifies all of them. This usually includes producing engineering and qualification units of hardware components (Section 28.8) for testing, including destructive testing for some parts. It also usually includes integrating all of the engineering or qualification units and the corresponding software into a testing version of the entire spacecraft in order to verify the entire integrated system.

Verification in Phase C typically includes verifying the human interfaces in the system. Can an operations team use the ground systems to accurately control the spacecraft, using simulated telemetry showing the spacecraft in different conditions (including off-nominal conditions).

Phase C is typically divided into two parts: the first part for completing all the designs, and the second part for implementing and producing the components. The Critical Design Review separates the two parts, where all the designs are checked.

I have seen the Critical Design Review milestone cause confusion: how far should work progress before the CDR? What is the boundary between “design” and “implementation”? For hardware components, such as an electronics board, engineers work on the board design: the layout of the components and traces that will be fabricated. The NASA CDR definition ([NPR7123, Table G-7, pp. 113-4]) indicates that the CDR should include “integrated schematics” and “fabrication, assembly, integration, and test plans”, which would indicate that the board design is complete. That the document also indicates that the CDR and Production Readiness Review are often coupled lends credence to the interpretation that the CDR reviews the board designs.

If this same interpretation were applied to software, it would imply that the software would be essentially complete by CDR. Software source code is the equivalent of electronics board design: while it is thought of as implementation, it must be processed through a build system to produce the actual executable software, just as a board’s design file is used to manufacture the boards.

However, the NASA Systems Engineering Handbook states that the CDR for a software component should occur “prior to the start of coding of deliverable software products” [NASA16, Section 3.6, p. 29]. In other words, the documents appear to disagree, though NPR 7123.1 is presumed to have precedence.

Further, software is often developed iteratively, implementing one version after another, each version adding some amount of functionality over time. Some lower-level functionality is left as a mockup, perhaps not even fully designed, until some of the higher-level integrated functionality has been implemented and verified (the idea of integration-first development (! unknown reference XXX), done to reduce risk as quickly as possible). Software development best practice also has verification proceeding continuously throughout implementation, with feedback to the implementer as early as possible. This often implies having some of the hardware components built and available for testing the software before the software is completed.

An official answer to how a team should resolve the discrepancies and interpret the CDR for a NASA project will have to come from the relevant NASA authorities.

However, in practice, I have found that focusing on the review before implementation is more useful for components in the upper and middle levels of the breakdown hierarchy. For example, this might include the major components with a subsystem, such as power distribution or generation within the electrical power subsystem, or attitude control algorithms in the guidance, navigation, and control subsystem. Components at these levels realize the important relationships between components in the system structure (Section 12.2) and the way components work together to produce emergent properties (Section 12.4). Analyzing these designs allows one to check whether key system behaviors will be met, and that properties like safety or reliability are handled correctly. These are the properties that are difficult to change if the implementation is found during verification not to meet them. The design and implementation of low-level components should be reviewed, but as long as there is an obvious, low-risk approach for them their review need not block the design reviews of the system as a whole. This interpretation, of performing the critical design review before implementation, means that the team is then free to implement software components incrementally if that is the best approach for that part of the system.

There are three reviews in Phase C:

The NASA Phase C maps to completing the development phase, the acceptance phase (Section 28.9), and the system production phase (Section 29.1) in the reference life cycle.

The CDR milestone maps to a slice through the system and component development phases, at the end of the design step for most or all of the components. The PRR for a component is equivalent to a review at the end of the production unit development step (Section 28.8). Note that the reference life cycle has a manufacture and deployment check milestone in the acceptance phase; this applies when the entire system is manufactured together, rather than the model implied in the NASA life cycle where different hardware components go to production individually. Finally, the SIR is equivalent to the deployment readiness review that is at the beginning of the deployment phase (Section 29.3) in the reference life cycle.

31.2.6 Phase D—System assembly, integration, and test, launch and checkout

Phase D covers the work between the end of designing and building all the parts and having a spacecraft on orbit ready to begin its mission proper. This includes assembling the spacecraft and ground systems, and verifying that they work (and work together). The verification typically involves testing the assembled spacecraft in vacuum, under strong vibrations, and in thermal environments equivalent to what it is expected to handle in flight—but not testing beyond those levels, in ways that might damage the vehicle. After testing, the team proceeds onward to integrating the spacecraft with its launch vehicle, final preparations, launch, and starting operations on orbit. The team on the ground finally checks the spacecraft out before declaring it ready to begin its mission.

Some missions build a second copy of the spacecraft to be used on the ground for debugging issues with the one in flight and to test possible commands before sending them to the operational spacecraft. The duplicate is typically assembled in Phase D. It might use qualification units for hardware that were used for testing in Phase C, rather than flight-ready units.

There are several reviews in Phase D. All of them are final checks that some part of the mission is ready for taking an irrevocable step. These include:

The NASA Phase D maps directly to the deployment phase in the reference life cycle. It takes in manufactured components and procedures, assembles them into a working system, tests that it has been assembled properly, and starts it in operation. The milestones in the NASA Phase D are different from the deployment phase milestones mainly because they are specific to launching a spacecraft.

31.2.7 Phase E—Operations and sustainment

In this phase, the team operates the mission through its end.

There are two kinds of reviews that occur in Phase E:

Phase E is equivalent to the system operation (Section 29.5) and evolution (Section 29.7) phases in the reference life cycle. The Decommissioning Review is equivalent to the decision to retire the system at the beginning of the system retirement phase (Section 29.9).

31.2.8 Phase F—Close out

The final phase in the NASA life cycle involves retiring and disposing of the flight systems, retiring or releasing ground systems, archiving mission data, and closing out the project.

There is one review identified in the NASA life cycle:

This phase corresponds to the system retirement (Section 29.9) and project ending (Chapter 30) phases in the reference life cycle.

Sidebar: Summary
  • The reference life cycle is a foundation for developing the patterns for one’s own project.
    • Tailoring includes adding details and defining procedures.
  • There are several questions to consider when tailoring the reference life cycle.
  • A detailed mapping from the NASA life cycle to the reference shows how to fit with the NASA approach.