Making systems - 34: System concept

34.1 Introduction

At the very beginning of a system development project, there is generally at most a rough idea of what the system should be. The understanding of the system objectives is too vague to launch into development or writing tests right at the start.

The concept comes after working out the purpose for the system: who the stakeholders are that have an interest in the system, and what they need.

The system concept is the high-level summary of the system that the team will build in order to satisfy those needs. It defines—in a coarse, high-level way—the structure and behaviors of the system, who will use it and how, and how the team will approach building that system. It defines the scope of the system, that is, where the boundary lies between what is in the system and what is outside it.

The exercise of putting the concept together involves exploring different ways that the system could be organized. It is the time to investigate multiple high-level designs to see which ones best satisfy stakeholder needs, and to learn about the problems to be solved building the system. It is also the time to build up an understanding of major risks that could arise.

The concept that is selected at the end of concept development is one of the most important, near-irrevocable decisions that the project will make. Once the concept has been decided upon, the rest of the project’s work is to realize that decision. This means that the choices made in the concept must be well-informed and made carefully. At the same time, the concept does not have to be perfect; it will be revised as the project moves forward and the team finds the inevitable problems with the initial concept. Instead, the initial concept needs to be correct enough to bring the risk of later surprises down to an acceptable level.

Concept development formally ends when a concept review approves the concept documents. At this point the documents are baselined, meaning they are now stable and can be used to develop the system. Any further changes to the concept will require additional review and a new baseline.

After the concept development phase, the record of the concept remains important. Because it is the definition of the high-level system structure, it informs how the system is decomposed into components (Section 11.3). It defines high-level structure and interactions between those components (Chapter 12). The concept and the system purpose are the basis for system specifications (Chapter 36). All this information that follows from the concept should be traceable back to the concept.

Later in the project, the concept provides a guide to new team members who need to learn about the system before they start working on details. It serves management as a definition of the project’s goal, allowing the team to check whether they are designing and building what they intended to build.

34.2 Why build a concept?

There are several reasons to put the effort into developing the system concept.

First, the concept provides guidance for all the project’s later work. All the specifications, designs, and implementations will be about building the system defined in the concept. The concept helps unify all that effort toward building one consistent system. In the absence of a well-structured concept, the team will be taking undue risk designing and implementing parts of the system.

The finished system concept provides education for people joining later. It is a high-level summary of how the system is organized and functions, and new team members can use it to learn how the parts of the system fit together and how the system will be used.

The work of developing the system concept is an exercise in learning about the problem. What past work has been done that this project can reuse? What different ways are there to satisfy stakeholder needs? What major unsolved problems might lie ahead?

Building the concept is an opportunity to check feasibility. It is the time to see if everything that the customer wants is in fact possible—and if not, this makes that apparent and can be the trigger for negotiating something feasible with the customer.

Finally, the system concept provides an opportunity to validate that the team properly understands stakeholder needs. Building up the system concept requires examining stakeholder needs in the purpose in detail, and doing so can reveal inconsistencies and gaps in what the stakeholders have asked for. This becomes an opportunity to work with the stakeholders to clarify the system purpose. When the concept is complete, the team can share it with customers and other stakeholders to see if it meets their expectations.

The concept also supports the relationship between the project and customer by defining what the team will deliver. For projects working under contract, the concept informs the deliverables that both parties agree to. When a stakeholder has agreed to a concept, there is a clear basis for determining whether requests later on represent a change to the agreed-upon concept or not. This can lead to fewer difficult disagreements with a stakeholder.

A project that does not put time into developing a system concept runs several risks. There is a good chance that the parts of a system that they develop will not fit together into a coherent system because there is no overarching goal that the parts fit into. The project is likely to miss important alternate design approaches because they move too quickly toward some one design. There is also a good chance that whatever system they build will not meet stakeholder needs because they do not validate their approach early, leading either to expensive rework or a system that fails to meet needs.

One project I supported jumped into developing a few core technology components of the system they were building without taking time to think through the big picture for the work. The result was that when they began a couple years later to articulate what the whole product system might be, they had significant gaps in what they had built; they would need to re-engineer some of the major system components. Perhaps more important, they had developed institutional ways of thinking that made it hard for them to see where they needed to make changes for the whole system to work well.

In another project, a set of regulatory agencies developed a body of regulations and supplementary material about a kind of system that should be built. The result of the exercise was only an EU Implementing Regulation and a document expanding on those regulations. The regulation and commentary provided many details about the components in the system, but they did not record the big picture of the system: what behaviors as a whole the system was to provide, and how the components should support those behaviors. Instead each organization interpreted the regulations as best they could, and we found that different people had widely divergent understandings of what the components should do or the behaviors of the system as a whole. We spent a lot of time trying to reverse-engineer the concept from the hints in the details, and in the end it is unknown whether the concept we developed matched the regulators’ intent.

34.3 What a concept is and is not

I discussed what the concept includes in Section 28.4. The concept is a high-level view of what the system will do and how it will do it. It presents both the external and internal views of the system. The external view documents its behaviors and interfaces as seen by users outside the system. The internal view sketches the major parts and their behaviors that achieve the externally-visible behaviors. The scope—the boundary between the internal and external—provides a boundary to the system.

The concept therefore includes several kinds of information. These include:

The scope of the system (Chapter 10). This defines the boundary between the system and its environment. The project is responsible for building things inside the system, and has no responsibility for things outside (though sometimes it can influence the system’s environment). All of the system’s externally-visible behaviors exist at the boundary.
A definition of the external environment, especially of who will interact with the system.
A definition of the scenarios and behaviors that these external users will see; how the system will react over time to what they do and to other stimuli in the environment.
The interfaces that cross the boundary between the outside environment and the system. Some interfaces will be obvious, like levers or displays. Many other interfaces are less obvious: that energy will flow into the system, aircraft skin will affect airflow, or that people can observe something flying.
The major components involved inside the system, and how they interface with each other and the external environment.
Major behaviors of the components, in terms of scenarios that the system is expected to support.

This is a reductive way of looking at the concept, focusing on a breakdown by parts. This can lead to definitions of components or interfaces that don’t quite work together to meet the system’s objectives.

The concept should include a second, operational perspective on the system, focused on use cases or scenarios. This perspective shows how different externally-visible actions flow from a system user, through an interface, through components, and back through interfaces to users.

One can use the two perspectives to check the accuracy and completeness of both. When tracing activities through the operational perspective, each step of the operation should have a corresponding behavior in the structural perspective. Any operation that crosses the system boundary should be supported by an interface at the boundary. All the external behaviors in the structural view should have a corresponding flow of operations, and every operational flow should reflect some listed external behavior.

The information about the structure and operations in the concept is the outcome of decisions made by the team. The concept should therefore be backed up by rationales that describe why the team made the decisions recorded in the concept, and the arguments for why this particular system concept will meet the system’s purpose (that is, stakeholder needs).

Many projects will also include information about how the system’s development can proceed: some ideas about the order different parts should be developed and tested, the organizations and resources that will be part of the project, and some major milestones that might measure progress.

At the same time, the concept is not the complete specification of the system. It is just a sketch; it should not be very detailed. Work on the concept should stop when it appears likely to cover the most important use cases and stakeholder needs, is likely feasible, and appears to be self-consistent. It should have explored the system enough to reveal unexpected constraints or design risks, without necessarily solving all of them. It should not be very detailed; part of its purpose is to prepare for working out the details.

I have seen many projects try to turn the concept into the full system design. In doing so, they have bypassed important review steps and have produced an unwieldy document that people can’t follow. They have also produced a system design that has many hidden gaps or errors, and they have skipped the design and evaluation steps that would have found those.

As a project progresses, the concept’s role changes from being the initial possible design to being a high-level guide to the system. It becomes a way for new people to learn about the overall structure of the system. In this role it needs to summarize, rather than go into detail. It needs to be an accurate guide, which means that the team must update the concept as design and implementation proceeds. Later design steps are likely to find problems with the concept as initially developed, or find better ways to do something.

34.3.1 Concept versus CONOPS

I use the general term “concept” in this work as opposed to the term “concept of operations”, usually abbreviated CONOPS. In many organizations, these terms are be used informally and interchangeably.

Formal definitions of each can differ. For example, the NASA Systems Engineering Handbook [NASA16, Section 2.3] talks about a concept of operations as being a scenario of how a mission system can operate, and treats it as one part of the overall concept. The NASA model separates these operational scenarios from the work breakdown and mission architecture.

By comparison, I use “concept” in the broader sense of including system structure, users, usage scenarios, and interfaces.

34.4 Developing the concept

The question is, then: how does one come up with the system concept?

I will use a simplified version of the project I discussed in Section 4.1 to illustrate techniques. This was a NASA technology demonstration mission that was intended to show how multiple small spacecraft could work together to perform science missions. The early mission analysis identified two key technologies: the ability to operate the collection of spacecraft without having to communicate directly with each one from ground stations and the ability to communicate between spacecraft (“cross-linking”) to share commands and data throughout the collection. As with many NASA missions, the initial parts of the concept were worked out by an office dedicated to developing new mission ideas, with the initial concept handed to the project team to finish out.

34.4.1 Incremental development

Some projects can work out the initial concept quickly from start to finish. Other projects will require time and multiple iterations to develop the concept.

The amount of effort required to develop the concept depends in part on the project’s scope and on how much has already been determined. When a project is responding to a customer’s request for a proposal, there are often many constraints that narrow down the possible system system concepts. When a project is evolving an existing system, there is often only a narrow range of acceptable solutions. On the other hand, when a project is creating a new kind of system there are fewer answers already determined.

Working out the system concept is a process. The solution does not usually arrive in one flash of insight. The team should expect to work out the concept in pieces, and to backtrack over parts that once seemed good and settled.

This is an experimental process. It involves building up a set of hypotheses about what the system should be and then testing them. In many projects, the team will develop multiple partial concepts and discard them as they are evaluated. The team might develop prototypes or proofs of concept for key items to see if they are feasible. The team might split into groups, each working on a different, competing approach. This work of evaluating the possibilities is an essential part of the process.

Concept development can end when the concept is likely feasible, likely meets stakeholder needs, and is agreed upon within the team.

The concept is likely feasible when it is unlikely there are hidden gaps or technical surprises, and the system parts can likely be built (or, better, there are solution approaches for each part). High-risk aspects of the concept have been identified, investigated, and there are likely solutions.

The concept likely meets stakeholder needs when the team can show that all of the most important needs have been addressed satisfactorily. The concept might not yet address all the minor needs, but it is likely that the concept will do so.

The concept provides the common vision for the work the team will do as it designs and implement the system. Everyone on the team should be in agreement on the concept, with no misunderstandings. The team will need to put some effort into ensuring that everyone involved shares this common understanding, and possibly correct ambiguous parts of the concept documentation. It is important that no one on the team has a separate agenda that does not match the system concept. I worked on one project where one of the sub-teams was focused on using the project to develop a radio component that they had been wanting to build, even though that component did not meet the actual system needs.

The team makes tradeoffs when deciding that the system concept is done enough to move on to specification and design: completeness versus making progress, risk of making poor decisions versus just getting on with it, risk of stakeholder acceptance or not. There is no recipe for deciding how to make these tradeoffs.

The concept does not have to be complete and finished before part of the team begins specifying and designing some system components. Some people can begin working on parts of the system that are clearly needed without waiting for every bit of the concept to be worked out.

Proceeding to specify and design some parts of the system leads to three risks. First, there is the risk that further concept work will reveal an unexpected problem with the part of the concept people are working on, leading to lost effort when that part has to be redone. Second, proceeding on part of the system can create momentum to continue with a poor concept or design choice just because resources have already been invested in it. Finally, starting on specification and design can lead to a perception that the concept is finished even though it is not. This can lead to people being redirected away from further necessary concept work, leaving problems undetected until much later in development.

At the same time, starting some specification and design work can be an appropriate choice when the risks can be identified and contained. If one part of the system concept appears certain while other parts are not yet worked out, and there is reason to believe that those other parts will not invalidate the worked-out part, then part of the team can move forward into specification and design without undue risk—though there will still be some risk of further concept development finding a surprise.

One possible outcome of concept development is that some aspects of the system are just not ready to be developed. The stakeholder needs might not be well enough understood to make a choice among alternative concepts, or some part of the system might need more innovation than the project can afford at the time. Sometimes the best approach is to decide to only proceed to build a part of the system right now, and plan for future evolution as technology or market information develop. See Section 34.10.3 for more discussion on this.

Sometimes the result of the concept development effort is finding that the project should not proceed. In that case, the reasons for not proceeding should be documented and reviewed, the decision to end the project made, and the team can move on to something else.

34.4.2 Determining scope

In most cases, much of the system’s scope can be inferred from the system purpose. The purpose usually defines who will use the system (who are thus outside it) and the major functions the system is to perform (and thus inside the system).

There are usually some grey areas, however. Customers may quietly assume that someone will provide some capability and not talk about it. Maintenance or operator activities, for example, are often omitted; are those who do these tasks part of the system, with roles to be designed, or are they like users that sit outside the system?

In the example project, all the explicit stakeholder needs were focused on performing a science mission, including a definition of the kind of sensor to be flown and the measurements to be taken. There was also a focus on communication to and within the collection of spacecraft. The needs included a rough maximum budget and timeline.

The project purpose did not, however, address how the science operations would work. It was left unstated how science activities would work, how investigators might be involved, or what would happen with sensor data when received. This meant that it was unclear whether these functions should be included within the system or not.

34.4.3 Determining environment

The environment consists of everything outside the scope of the system that the system interacts with in one way or another.

The whole environment environment includes different kinds of things, including the physical environment in which the system resides; the information environment, how information is supplied to the system or used; the user environment of people who interact with the system.

The physical environment includes the place where pieces of the system reside. This can include atmosphere, dust, heat, humidity, impacts, vibration, radiation, and electromagnetic waves. It also includes what the pieces of the system are attached to or where they are placed: rooms, racks, brackets. It can include aspects like the accessibility of a system component for security, such as how a room is protected.

The energy environment is related to the physical. This includes how energy enters and leaves the system: electrical energy, hydraulic or pneumatic energy, mechanical forces, or heat.

The information environment includes the information that is sent to the system or received from the system. This might be other systems to which this one is connected through networks. It might be information that is provided to other systems through things like signal lights or physical gates.

The user environment records who will use the system, and what they will expect it to do. This typically lists who the people are—or rather, their roles—and what they do with the system. The user environment also includes what kind of interface the people will have, whether a typical computer system or lights, sounds, and levers. This can be the place to record what the skill level of the users and the task environment. A quiet office with a laptop is quite different from the flight deck of an aircraft.

Systems may well have more kinds of environment. In general, if something is outside of the system’s scope and it can affect how the system functions, it should be considered part of the environment. For example: government system operations can be affected by the organization that funds them, and commercial systems can be affected by the training and oversight budget that the company using the system provides.

Working out information about the environment is related to working out the interfaces between things in the environment and the system.

34.4.4 Developing system structure

Working out the broad structure of a new system is an exercise in creative problem-solving. Many, many people have published and presented material on solving problems; I do not mean to reproduce or summarize that work here. Polya’s work [Polya57] is a classic work on this, and I recommend it as a foundation for problem-solving in general. This chapter adds some techniques specific to working out a system concept onto that foundation.

I have taken a reductive approach to defining and understanding systems in this work. This means that a system can be understood in terms by breaking up one thing into smaller parts, and reasoning about how those parts combine to produce emergent properties. (As noted in Chapter 6, there are non-reductive ways to design and understand systems; however, all the ones I know of are suited more for machine development and not for human understanding.)

This approach means that developing a system concept amounts to working out how to break the whole system into a few high-level parts, along with how those parts interact.

A potential parts structure is appropriate if it meets stakeholder needs. Tracing out how actions and information flow through the system in order to effect externally-visible behaviors is one of the essential checks for a correct or complete structure.

There are several problem-solving techniques that can help guide one toward developing a system concept. Most of these are common to many different kinds of problems; a few are specific to developing system structure.

In all these techniques, bear in mind that developing the concept is not a smooth progression toward a clear end goal. The work is more like searching through a maze, where some approaches will appear to be promising for a while until someone finds a flaw. There will be many potential approaches to different parts of the concept, and often none of them are a perfect fit to the need.

Be clear about the problem. The first step to developing system structure is to understand the problem at hand. Understanding the system purpose is a big part of understanding the problem; that is why understanding purpose comes before working out concept.

The next is to really understand what the system should do and be—not just knowing what’s in the purpose, but getting to understand the behaviors or functions it should provide.

I find that this usually means putting together a list of clear usage scenarios. A usage scenario can be a statement like “user X does action Y; the system responds with result Z”. This list doesn’t have to be exhaustive, but it should cover at least the most important behaviors in the system purpose. It also should not be too detailed at this stage of the work. The purpose is understanding, not complete specification.

I then take each one of the scenarios and try to understand what they mean. I try to imagine what each one might involve inside the system by walking through the flows of information or control to see what different components might need to do or what specific functions might be needed. This exercise often reveals gaps where there is a problem to work out. In the small spacecraft example, one scenario was sending a command for all the spacecraft to take some sensor measurement at the same time, having them take those measurements, and get the results to someone on the ground who would use them. That scenario has implications: a command sent from the ground can get to all spacecraft; spacecraft know what time it is accurately enough to take the measurements; each spacecraft can record its measurements; there is a way to collect and transmit those measurements to the ground. Stepping through the activities involved in this scenario helped me at the time to learn much more about how the system had to work, and thus how it should be structured.

As people work out potential solutions for how the system might be split into major components, these scenarios are helpful in evaluating the solution. One should be able to take the scenarios and show how they will cause activity to flow through the components. In the example, I worked out a potential structure for spacecraft software functions involving a communication component, a routing component, a message storage component, and a command execution component. I could look at the scenario in the previous paragraph and trace out how each of these components would receive input, take action, and send results to other components. Being able to trace that flow of activity gave me confidence that the component structure was at least feasible.

Besides looking at specific scenarios, systems often have modes of operation where the expected behavior is different. This can help to group usage scenarios and to suggest whether the list of scenarios is complete enough.

For the small spacecraft project example, the system would have a few operational modes. These included:

Pre-launch, when spacecraft are given final checks, final parameter settings, and installed into the dispenser on the launch vehicle.
Launch and deployment, when the spacecraft must remain inert.
Startup, after the spacecraft has been deployed from the dispenser and turns on for the first time on orbit.
Monitoring, when ground systems monitor the spacecraft for correct operation and fix problems when they occur.
Testing, when ground systems send commands up to one spacecraft and check that the commands are distributed to all the others, and the reverse path to collect results.
Science operation, when ground systems instruct the spacecraft to perform data measurement using the communication framework that has been tested.
Off-nominal operation, when one or more spacecraft are not functioning as expected and should be fixed.
Deactivation and disposal, when the mission is ending and the spacecraft are made safe to remain in orbit until their orbits decay and they burn up on reentry.

Working through the scenarios and functions often reveals behaviors that were not stated directly in the system purpose. In the example system, not all of the operational modes were apparent at the start. The need for an explicit deactivation and disposal mode was caused by US requirements for ensuring that a space flight mission does not create orbital debris that would pose a hazard to other spacecraft. Many behaviors needed during pre-launch, launch, and startup come from requirements on launch vehicle safety.

Break the concept into subproblems. The next step is to begin to divide the system into parts. The goal is to work out all of the high-level components in the system. This is where creativity is required in working out a system concept.

Each part should deal with a separate concern. It should deal with one piece of system function; it should not try to mix dissimilar functions together. Section 11.2 described criteria for dividing parts of a system into components: singularity of purpose; weak coupling with other things compared to within the component; and ability to replace one design of the component with another.

Many times one approaches this goal piecewise, in small steps, because the overall problem is too big. Polya calls this “Hunting for the Helpful Idea” [Polya57, p. 34]. The idea is to explore the problem to find some ways to begin tackling it. This can be by finding a similar problem that has already been solved and applying it to the problem at hand. It can be by drawing out part of the problem and looking for information that will guide toward a solution. One can tackle just one part of the system, ignoring other parts for the moment, in order to find a partial solution that might be combined with other partial solutions. Sometimes looking at things that constrain a solution can help focus attention on things that will help.

Finding similar problems. In some cases, there are common structural patterns that provide a guide for how to decompose a system. Space flight missions, for example, almost always decompose into a launch segment, space segment, and ground segment; the ground segment in turn decomposes into communications, operations, and mission or science operations. Most fixed-wing aircraft are organized around common patterns: body, wings, control surfaces, power plants, landing gear, and so on.

In other cases, there might be a behavioral pattern that applies. When dealing with communication protocols, there is a body of knowledge about how to avoid common performance problems (rate control, avoiding head-of-line blocking) or how to achieve security properties (authentication exchanges, nonces, cryptography).

Using common patterns is a benefit when the patterns match the system purpose. Common patterns can, however, impose a bias when the patterns do not quite match what the system needs. For example, using the common patterns for fixed-wing aircraft structure are not a good fit when the best solution is a blended wing body structure, which does not have separate body and wing structures.

Sometimes there are common ways to approach a problem. There are analytical methods for designing system safety and security, which I will deal with in Chapter 46. These provide a specific set of steps to follow to work out specific system needs.

For the example mission, we could draw on the history of many cubesat-class spacecraft [Cubesat22] that have been flown. We reused the basic system structure, consisting of a frame, electrical power system, flight computer, attitude determination and control, communications, and so on. Communication patterns among the spacecraft were based on previous work in the early Internet, when nodes were not always connected.

Sketching out part of the problem. Diagrams are powerful tools for exploring system design. I explore many system ideas by drawing block diagrams. As I write this, I have been working on a concept for a possible new system. I talked with the stakeholder to understand the key functions they expected; this revealed who the major users would be and what the system might do for each one. I then started imagining a set of internal functions the system might have to perform these functions, and sketched the way data might flow among those internal functions. This has begun to reveal possible ways to break those internal functions into high-level components. Of course, this led me to only one possible way to decompose the system; I later tried sketching from a different point of view and saw different things. (I discuss the value of multiple viewpoints below.)

For the example NASA mission, the highest-level hardware components were part of the definition: a collection of small spacecraft, plus ground systems to talk with them and a launcher to get them all on orbit. Sketching how the spacecraft would move relative to each other (a problem in orbital dynamics) and how they would need to communicate with each other was useful for trying out different communication strategies.

Trying many approaches. I find that the first idea I have for a solution is often not the one I end up with. In some cases I have started sketching approaches from scratch multiple times, each time starting with a different system function and seeing where that leads.

In one one project on UAS traffic management, my team needed algorithms that could generate a flight plan. The trajectory needed to avoid obstacles and stay away from other nearby traffic. We investigated perhaps a dozen algorithmic approaches to the problem; some were quite simple, others more complex. We looked at different path planning algorithms in the literature and evaluated how they would perform for our specific problem. In the process we learned more about the actual requirements for the path planning algorithm. The final choice was different from any that we knew about at the beginning.

This example is typical. We looked for different algorithmic approaches that people had developed. We evaluated some of them on paper; we prototyped others to see how they performed. We threw away several prototypes when we found use cases they would not handle, or when they did not have the needed performance.

Finding clusters of behavior. Some systems will have groups of similar behaviors or functions. Such groups are often a clue that they will share a common component.

When I was working on a system different from the example above, there was a clear need for sharing multiple kinds of information among many different nodes, and those nodes mostly could not communicate directly with each other. Seeing that multiple system functions shared this behavior led me to look at a common data sharing component.

Tackling part of the solution. For a system that doesn’t follow existing patterns as a whole, sometimes there is one part of the function that one can figure out. One can figure out a possible approach for one piece, then trace out system behaviors using that component. This can in turn reveal clues for how to tackle other parts of the system.

For example, in the example NASA system, I had an idea for how to address and forward messages between spacecraft. This involved identifying which spacecraft (one or multiple) need to get or send some information; it involved creating logical “channels” for senders and recipients and addressing messages to channels. I then started tracing what would happen when a command was generated on the ground, addressed to some channel. This led to thinking about how to schedule communication sessions between spacecraft, how to store and manage messages that are being shared, and so on.

This was only part of the solution, of course. It addressed nothing about how to provide behaviors for how to start up a spacecraft after deployment, or how to address off-nominal behavior. But it provided a start, and slightly reduced the complexity of thinking about other communication behaviors.

Dependencies. Working out one system behavior can reveal other necessary behaviors that are implied. The first function depends on other functions happening; those other functions depend on yet more functions, and so on.

The example mission was expected to communicate between spacecraft and a ground station. The ground station would have one antenna to communicate with all the spacecraft. The spacecraft lacked the power to keep the transceiver turned on all the time. This implies that each spacecraft had to determine when the ground station was about to come into view in order to turn on their transceiver. Knowing this event implied knowing where the spacecraft and the ground station were. The ground station location could be configured before launch, but the spacecraft location was dynamic. It could be determined three ways: using a GNSS receiver, by modeling the spacecraft’s orbit and computing a location, or by keeping the spacecraft pointed toward the ground (nadir) and listening for a message from the ground station. Modeling the spacecraft orbit implied determining what the orbit is, either from observation (by GNSS, for example) or by being told by ground systems. Of course, being told by ground systems implies communicating with the ground, which creates a circular dependency. Keeping the spacecraft antenna pointed toward nadir and listening for a message implies determining which direction the ground is, pointing the spacecraft appropriately, and keeping the receiver powered up to listen for the signal.

In addition, the spacecraft was using a directional antenna and the radio signals were transmitted at low power. A directional antenna means that the antenna receives (and sends) signals more strongly in some directions than in others. This implies that the antenna needs to be oriented so that a direction where it sends and receives most effectively is pointed toward the ground station, and vice versa. Being able to point the antenna implies knowing the direction toward the ground station, bearing in mind that the direction will change as the spacecraft flies past it.

In other words, the design of the communication and electrical power systems created a need for spacecraft guidance and attitude control capabilities.

Looking at constraints. The space of possible solutions for a novel system can be very large. Being able to focus attention on a smaller set of problems can help.

I worked for a while on the basic designs for UAV traffic management (UTM) systems. (I discuss this work as a case study in Section 63.7.1.) These systems provide traffic control for UAVs (drones), keeping them separated in flight from each other and from manned aircraft, among other functions.

Working out a flight path for a UAV and then ensuring that the planned path will stay far enough away from other aircraft are two of the key functions of such a system. One fundamental question was: what is the best structure for these planning and checking functions? This decision shapes how much of the rest of the UTM system is organized.

In that project, we expected that the UTM system would need to handle many flights occurring in shared airspace, that there would be multiple kinds of UAVs with different performance characteristics, and that there would be multiple organizations flying those UAVs. Those organizations would be expecting efficient and fair use of the airspace. Some organizations would be designing and building their own UAVs, and they would regularly be introducing new capabilities for their aircraft.

The need to support multiple kinds of UAVs turned out to be an important constraint.

The process of computing a possible flight plan depends on where the flight is supposed to go and the capabilities of the aircraft to fly, hover, or maneuver. One UAV model is also likely to have different abilities to communicate with ground services or its operator than others. This means that computing a flight path can require a detailed model of the UAV’s capabilities. In addition, different operators need different kinds of flight paths: one operator may want their UAV to fly from point A to point B, while another wants their UAV to fly a back-and-forth pattern covering a farm’s field.

There was an early design choice. Should the UTM system provide centralized flight path planning, or break up the planning into multiple systems? In the centralized approach, one common component would take in requests for a flight plan and compute a plan, in the process consulting databases of what flight plans were already approved and what aircraft were already in flight. This would have the advantage of assuring that all flight plans would be properly checked before being returned, and it could lead to computational efficiencies by coupling planning and databases. A centralized approach, however, would have a serious problem: it would need to know the performance model of every UAV and every kind of flight path any operator might want, which would be infeasible—especially if some operator wanted to innovate.

The constraint that the UTM system must support multiple kinds of aircraft and flight paths made centralized flight planning infeasible. We quickly focused attention on designs that made computing flight paths the responsibility of individual operators or of services they used, and investigated other ways to ensure that flight plans were checked properly. We did not spend any more time on centralized computation.

Multiple viewpoints. When one person works on a design, they are likely to find some partial solution and then proceed from there. This can make it harder to think of different approaches.

Having multiple people work on the problem independently can find different starting points and come up different design approaches.

The different approaches can be evaluated and compared. They may have different strengths and weaknesses, which are sometimes only discovered when one analyzes how well each approach satisfies system needs. Sometimes knowing strengths or weaknesses will prompt someone to improve one of the solutions. Sometimes the solutions can be combined to yield something better than either.

Evaluation. A concept is only good if it is fit for purpose. That means satisfying the stakeholder needs recorded in the system purpose. Each concept developed is checked against those needs to see how well it does, and alternative concepts can be compared based on how well they satisfy the purpose.

There are several kinds of evaluations to be done.

Checking that the system provides the functions needed by stakeholders.
Evaluating the likely performance or physical characteristics of the system: mass, resilience, expected environments, or operable lifetime. Aircraft concepts might address an estimated speed, range, and payload. Spacecraft concepts might address likely attitude control, delta-v capability, or lifetime.
Gathering evidence that the system can be safe, secure, reliable, or have similar high-level properties.
Determining roughly how much technical risk or invention will be needed, or alternatively how much can be re-used or acquired off the shelf.
Making initial rough estimates of the time, cost, and effort needed to develop the system.

Often, a back-of-the-envelope check is enough evaluation. Sometimes a simple model or prototype is needed to check out part of the concept.

Almost none these aspects can be evaluated with much certainty at the concept level. Most will require further detail that will come later with decomposition and design. The standard for these evaluations at concept time is merely whether it is plausible that the needs can be met.

When some parts of a system concept can be matched to a comparable existing system, it is generally possible to make quantitative estimates of cost, behavior, and other properties can be made with moderate accuracy. The advice from McConnell on estimation applies here: be honest about how much is and is not certain, and refrain from making precise commitments when the information does not justify it [McConnell09].

Evaluating the concept should reveal where there is technical uncertainty or programmatic risk. When one high-level system component will require innovation, that represents uncertainty. This information should be gathered and recorded along with the concept itself.

The concept is an appropriate time to begin to evaluate safety, security, resilience, and similar properties. Design for these properties starts as the concept is developed [Leveson11, Chapter 9]. The system purpose should include the foundation for safety or security by identifying what should be protected, which enables identifying hazards in the concept. Once the hazards are identified, one can evaluate the concept to see how they can be eliminated or mitigated at the high level. I discuss the process further in Chapter 46.

The process of evaluating a concept leads to discovering the strengths and weaknesses of that concept, especially a partial concept. Finding weaknesses can suggests ways to improve to find a better concept.

The concept evaluation should also produce evidence that the system purpose (stakeholder needs) has been met, in the form of traces from the items in the purpose to the parts of the concept that show that those items are addressed. This might be documented as a compliance matrix or table. The traces may need to be accompanied with some kind of argument for why or how the need is addressed. This kind of tracing is important when a reviewer or stakeholder checks the concept. This is essential, for example, when generating a concept that is for a response to a request for proposals.

Choosing between alternatives. There will usually be multiple ways to structure some part of the concept, and at some point one will have to make a choice between these alternatives.

During concept development, these decisions have a great effect on the later course of the project. Once people have been set on the course to specify, design, and build a system in a particular way, their effort has been spent and changing course to a different basic system structure means discarding that effort and the cost of changing the team’s direction (see Section 8.1.5 for a discussion on how teams have inertia).

In other words, these choices should be made carefully. Each alternative should be worked out to a similar degree of detail, and evaluated against the same criteria, as discussed in the evaluation section above. In some cases, asking for customer feedback on some of the promising alternatives can provide useful input. The project eventually needs to commit to one concept, and then stick with that decision—no factions trying to resurrect a different approach later.

At the same time, good decisions do not necessarily require high precision or detail in evaluation. The evaluation only needs to be good enough or thorough enough to be confident in the comparison between alternatives. If alternative A has ~10x the value of alternative B, one does not need more than one digit of precision to make a correct choice.

Biases easily creep into this kind of decision. It is important that the team evaluates each alternative on the same criteria. There is a particular risk in comparing alternatives based on a sum of weighted scores—so much for property A, so much for property B, and so on, with a weighting factor for each property. The scoring and weighting is fundamentally subjective in most cases, and it is possible to bias the results either by selecting arbitrary weights that predetermine the outcome or by inflating some scores for some approaches.

I was involved in one project that selected a basic design concept for an electric vertical take of and landing (EVTOL) aircraft, which would be able to take off and land vertically like a helicopter and also be able to fly horizontally at reasonable speed and range. The team enumerated a large number of options, with different numbers and arrangements of motors and wings. They evaluated them on the basic flight characteristics (speed, drag, efficiency) and on safety (such as resilience to motor failure). They eventually chose a particular design that did the best over all on these metrics, and the aircraft worked as expected.

Rationales and other information. Knowing why the concept is what it is can be as important as knowing what the concept is. Someone will need to revisit the concept in the future—to learn enough to make a change in the system, for example. The concept is likely to include the results of some decisions where it is not obvious why the decision was made as it was. The concept is also likely to contain structure whose purpose is not obvious, that took careful thought to work out. In that case, a component A may have some property that is important for component B, which interacts with A, to function correctly. When someone needs to implement or fix component A, they need to know that this property and relationship must be maintained for the system to work correctly.

This can be done by:

Annotated the concept with explanations to document some of this information.
Keeping a copy of the evaluations that have been done on parts of the concept and linking them to the appropriate parts of the concept.
When there has been a decision between alternative approaches, keeping a record of those alternatives and why some were rejected.

Other information is produced while working on the concept. This includes lists of where there are uncertainties and risk, and where specific technologies or off-the-shelf components can be used.

Completion. The work to develop the initial concept is complete when the concept can be shown plausibly to meet the system purpose. The uncertainties and risks discovered should be reasonable—none of them requiring invention of the implausible or miracles. The concept should be documented in a way that team members can read it to get a general understanding of the system they will be building.

Stakeholders review and validate the concept before it can be considered complete. This is a final check that the concept is correct.

The standard for completing a revised concept is higher. A completed revision also meets the system purpose and is readable as a guide to the system. A completed revised concept is also consistent with the way the system is decomposed into components, their specifications, and the further decomposition and design, so that it is an accurate guide.

The system concept should be kept up to date as the system’s design evolves because one of its purposes is as a guide or introduction to the system.

34.5 Artifacts

The concept development effort produces several artifacts: the concept itself, plus analyses, rationales, and initial lists of uncertainties and risks that have been discovered.

34.5.1 System concept

The system concept artifact has three main purposes:

To inform stakeholders about how the system will work at a high level, so that they can validate that the concept will meet their needs;
To educate team members about the high-level system structure, and why the system has that structure; and
To guide the team as they develop the artifacts that build on the concept: the system specification, the top-level system design, and the top parts of the component breakdown structure.

In other words, the concept is for teaching people about the system’s big picture. However it is organized, it must provide the information in a way that the expected audience (both new and experienced team members, stakeholders) understand. The concept must be accurate and complete to meet this aim. If the concept presents information that is different from how the system is actually built, people will be misled and make mistakes. If the concept artifacts omit some necessary information, people will not know there are things that need to be built or understood--again leading to mistakes either in evaluation or system-building.

At the same time, as I have mentioned before, the concept should remain the concept, not the whole design. It should be an introductory guide, not the specification. Doing otherwise usually means that the concept takes longer to finish than is needed, and that people end up deprived of the high-level guide.

There are three ideas that the concept artifacts convey: the system’s scope, its structure, and its behavior (Section 34.4). Each of these are connected, but need to be treated fully on their own as well.

The system concept can be recorded in many different forms and media. I have seen it developed as a prose document with some diagrams, as information maintained online in a wiki or systems engineering tool, and sometimes as a few diagrams with explanatory text. Semi-formal notations such as SysML can be helpful for documenting parts of a concept that are best explained graphically. (I have not found a purely graphical approach to work well; a couple projects I have worked on have tried to use SysML or UML as their primary means of documenting a concept, and the result was not understandable by most people who needed to use it.) Diagrams seem to work better when embedded in a textual framework that guides the reader through the information. The DODAF defines a standardized collection of viewpoints to record aspects of a system concept [DOD10].

A good concept document is often anchored by the “big scary picture”: one graphic that shows the major ideas in the system. The OV-1 diagram in the DODAF standard [DOD10] is one reference example.

Many projects record the concept in a single document. This document should be relatively short—a few tens of pages at the very most. The document is often structured as a reference to the system purpose, followed by the definitions of some major use cases, system scope or boundary, the high-level component breakdown, and finally discussions of how these components work together to support the use cases.

For the example NASA technology demonstration mission, I used a wiki system to document the system concept. The concept had two parts: the high-level system components (spacecraft, launch segment, ground segment) and a timeline for the mission. (I did not document the scope in the concept, because that had been defined in other mission documents already.) The timeline was divided into the mission phases. In each mission phase, there was a description of its purpose and a list of the major steps. Some steps themselves were broken down into timelines showing detailed steps. Each of the system components and timeline steps was written down as a wiki page, and there were extensive cross-links among the pages to help people find related information.

The system components part of the wiki was organized as a basic block diagram of the whole system, including the launch vehicle, spacecraft, and ground systems. Each component then had its own page with more information, with a description of what it would do or key functions it would have.

The timeline part of the wiki had one page for the overall mission phases, from pre-launch through decommissioning. Each phase then had its own page with details, including a cartoon of what would happen, a description, and list of important steps.

Finally, to repeat: the concept document should be short. I have seen many projects try to use the “concept” to work out and record the system specification. One can recognize when this has happened because the document runs to hundreds of pages, includes lots of details, and is usually abandoned shortly after system development begins. Instead, keep the concept as a high-level explanation. Let the details come in the system specification, which will be long, tedious, and written in stylized forms that are not easy for the uninitiated reader to understand.

34.5.2 Analyses

When it’s baselined, the initial concept should be likely feasible, likely mostly complete, and likely to meet the system purpose. I emphasize “likely” because at the beginning of a project that’s the best that can be done. As the project progresses, the concept should be maintained so that it matches the actual design and implementation, at which point it should be known to be feasible, complete, and compliant because the design and implementation are.

Feasibility of components.

Is there a model to follow for one of these? Is there anything unusual about this system that will cause deviation from that model?
If there is innovation needed, is there someone addressing it? Are they capable of finding a solution?
Where constraints are known, can they add up and be okay? For example, are parts that meet mass limits available? Communication components that can meet likely data rate needs?

For the example mission, we knew that we would build a fairly standard cubesat spacecraft. There are many components and designs available for those. We could build on the mission operations and ground systems from another NASA mission that part of the team had built before. There were multiple options for launch vehicles.

Feasibility of behaviors.

Is there a plausible story about how each behavior can work?
Where are there difficult things that might come up?

In the example mission, most mission steps are similar to those in other cubesat missions. However, we identified three potentially difficult activities: ground communication, power management, and crosslink communication.

Ground communication differed from previous missions in that there would be multiple spacecraft operating, and more than one might be in view of the ground antenna at the same time, potentially causing interference.

Power management differed because we had more components on board and could not operate most of them continuously. This interacted with communications.

Crosslink communication was the most difficult. Multiple actions had to go right for crosslink to work: basic ability to close the link, which depended on orbital dynamics; antenna design; and spacecraft attitude control. These depended on being able to point the antenna on both spacecraft so the other spacecraft was in view. That in turn required knowing where the others were, which had to be worked out from ground observations. We didn’t completely solve this problem before the project ran out of funding.

Key characteristics. These include system characteristics like safety, security, reliability, lifetime, or manufacturability. Each of these characteristics has methodologies for analyzing and designing the system. These analyses are most effective when started at the beginning of the project, so that all later work is informed by these concerns. Chapter 46 discusses how to design and analyze for safety and security.

Performing these analyses on the concept reveals capabilities that will be needed. A security analysis probably will reveal the need for some core security services like authentication and authorization, and these in turn typically involve some customer staff keeping track of who should and should not have system access.

Time, cost, and resources. Stakeholders usually expect the system to be developed within some budget and time. The concept is the first point to look at whether this is plausible or not. Any analysis done at the beginning of the project can only be plausible at most, because so many details remain unknown. The team must be careful that the results of these analyses are not treated as project commitments at the beginning of the project.

A project depends on other resources as well. Is it possible to tell what kind of people will be needed to complete the system? Are there kinds of test equipment that will be used? If vendors are likely going to be needed, are there vendors that could plausibly deliver on time?

34.5.3 Evidence of compliance

The standard for compliance is that all of the purpose is met, and that the system contains nothing extraneous. The system purpose should already list all of the identified stakeholder needs.

The evidence of compliance shows how the concept meets each of the stakeholder needs in the system purpose. This can be simple, like a table mapping items from the purpose to parts of the concept. Often this is clearer with an explanation of how or why some part of the concept ensures a need is met. Some items in the purpose won’t be addressed directly by the concept, such as workmanship or project management requirements; these should be noted as being addressed outside the concept.

Compliance also means arguing that there is nothing extra in the concept. This can be addressed by a mapping from each part of the concept to the items in the purpose it addresses. Sometimes the mapping won’t be direct; for example, a security need translates into a set of more specific objectives, and the concept addresses those specific objectives.

34.5.4 Rationales

The concept itself documents what the concept is, but not why it is the way it is.

Providing a rationale or explanation documents why choices were made the way they were. When someone comes along later to make changes to the system, they can learn about the original design thoughts and take those into account as they make changes. The rationale also records reasons that may not be apparent to later readers.

The rationale records subtleties in the concept. For example, when one part of the concept is intended complement another part, and the two parts need to be consistent for the combination to work. This situation is often not obvious from looking at either part, and adding a rationale will make the reader aware of the connection between the two parts.

Rationale is often written in prose, sometimes with diagrams. The information can be attached to parts of the concept, since the rationale is usually related to why some component is the way it is.

34.5.5 Uncertainty and risk

People often find uncertainties and risks while they work out a system concept. Other risks are identified while working out the system’s purpose. This information should not get lost—it should form the basis for later in the project when that information is used in planning.

The concept is complete when it is likely feasible and compliant, meaning that important risks have been identified, and most have been addressed. One needs to track risks and uncertainties to be able to check that the ones that matter have been addressed.

See Chapters 65 and 66 for how these can be tracked.

34.5.6 Configuration management

Because the concept provides a high-level guide to all the rest of the work in the project, people need confidence that they are looking at an accurate version of the concept. The concept will evolve as the project goes on, as stakeholder needs change, and as the team refines the system’s design.

This means that there will be versions of the concept that are in progress, currently baselined, and outdated. A baselined concept should be consistent with the matching system purpose artifacts and corresponding specification artifacts.

A work-in-progress concept should, therefore, get an explicit review and approval before being baselined, and its artifacts should be maintained in a configuration management system that reflects these versions.

34.5.7 Artifact maintenance

The concept will evolve of the course of a project. There are two reasons the concept might change once baselined.

First, the system purpose can change, and the concept changes to reflect the changed objectives. When a stakeholder requests a change to the system purpose, the process in Section 33.10 applies. That process includes evaluating a change request to decide whether to agree to it or not. Evaluating the request includes evaluating the effect on the system—what technical changes will be required, and the effort or cost involved in making those changes. Updating the concept is the first step in working out how big the change will be. In many cases a change to purpose and concept will be adopted and baselined together.

Second, the system will evolve as the team works into the details. They may find that some high-level design approach does not work as expected, and take a different approach instead. The team may come to understand part of the system better than at the beginning and find different ways to explain the high-level picture. In these cases the team should keep the concept up to date with the actual system so that the concept remains an accurate way to learn about the system.

34.6 Feedback to system purpose

In practice, a team does not work out the system purpose completely then turn attention to the system concept. Instead, they are likely to work on the two together. As they learn about objectives in the system purpose, they add necessary features to an evolving concept.

Information flows the other way, as well. As the team works out concepts for the system, they begin to understand what is possible and what is expensive. The team may find that some stakeholder needs are not feasible: they would require some capability that is formally impossible or that will require significant innovation that couldn’t be done in the customer’s time and budget. (I worked on one project where an executive asserted that the team could develop algorithms well known to be impossible.) This feeds back to work on the system purpose, and should prompt a negotiation with a customer about what is possible.

Developing the concept also leads the team to discover other potential requirements. For example, the stakeholders might initially make a broad statement about security (“we want it to be secure”). Working on the concept can prompt more specific questions to the stakeholders. What kinds of security hazards concern them? What are they willing to do for different levels of protection? These questions lead to updates to the system purpose, which will then match the concept.

34.7 Validating concept

The team will work to ensure that the concept satisfies the system’s purpose, as recorded in system purpose artifacts. The evidence of compliance, discussed above, records how the team believes that the concept matches the purpose.

However, this means that there is one step of indirection between what the stakeholders actually said and what is in the concept. There could have been misunderstanding in recording stakeholder needs, or there could have been problems translating the purpose into concept. For example, on one project, stakeholders used the term “real time” to describe one necessary feature. Unfortunately “real time” has multiple interpretations in different engineering disciplines: performing work interactively as opposed to off-line versus performing work to meet strict deadlines, for example. The system purpose included “real time” in its objectives; different concepts complied with different definitions. It was only when reviewing concepts with the original stakeholder that the ambiguous definition became clear.

Validation comes directly from the stakeholders when they exist. The point is to check that they understand and agree. In cases where a stakeholder does not exist yet, such as for visionary projects (Section 33.3), someone independent of the project acts as a proxy for future stakeholders.

Some stakeholders can review the artifacts that document the concept, as long as they have people on their teams who have the background to understand the media and language used in the documentation. When a concept is presented to a customer as a proposal, they will either read the concept directly or read a version translated to their language.

Other stakeholders will not read the concept directly. They may need to have the concept presented in language that they are comfortable using. Providing the material interactively so that they can ask questions can also help.

I have used three main approaches for validating the concept with those customers: a translated summary, presentations, and acting out scenarios—often in combination.

Summary documents, written using the stakeholders’ language, are more or less equivalent to the concept artifacts except that they are written for the stakeholder to understand. They are meant for the stakeholder to read on their own. Summary documents may leave out detail that a stakeholder will not care about, such as design details that matter for guiding the team but do not define system functions. The summary should, however, call out information that a stakeholder will care about but may not realize until it is pointed out. This might be an implication that the project team has discovered: having function A requires compliance with regulation B, or function A affects how function C can work.

Presentations are like summary documents, in that they are written for the stakeholder to understand and usually leave out details that the stakeholder does not care about. Presentations are interactive, where team members interact with stakeholders to tell them about the concept. The stakeholders can ask questions, and the team members can confirm that the stakeholders understand parts of the material. The team members can add extra explanation when the stakeholder needs it. Bear in mind that a presentation is an action performed by team members; a presentation is not a summary document written as a slide deck that a stakeholder reads on their own.

Acting out scenarios complements presentations. Team members prepare different scenarios that the system would experience; they and stakeholders can role play different system parts. The stakeholder can get an appreciation for situations that they might normally not think of. A colleague and I acted out some scenarios about how two organizations might negotiate a resource access problem. After we acted out a normal scenario, my colleague began to act out ways that one of the organizations could negotiate in bad faith. The stakeholder had had a mental model of organizations always behaving cooperatively; they appreciated how things could go wrong after acting out the scenarios. They then understood why we had features in the system concept to detect “cheating” and incentivize good behavior.

34.8 Reviews and approval

In the reference life cycle, concept development ends with a conceptual design review (Section 28.4). Passing this review means that the team is ready to move onward to system specification (Chapter 36) and design.

The team is ready to move forward when they and stakeholders have confidence that the concept:

Is likely complete enough;
Is a good solution;
Is feasible for the team to build; and
Likely complies with the system purpose, meeting stakeholder needs.

Two reviews are needed: an external review by stakeholders and an internal one by the team and others. The stakeholder review validates that the concept can meet their needs. This often requires the reviewers to look at analyses of the concept, not just the concept itself. The internal review checks both compliance with system purpose and feasibility.

A feasible concept is one that the team has or can get the resources and skills to do the work needed. Making this judgement implies that the concept is complete enough at least to name the areas of work involved.

A feasible concept is also one with plausible ways to implement all of the parts of the system. This does not mean necessarily that there are definitely solutions for everything; that will only be determined when the full system design is done. Some parts of the system can have uncertainties, but the concept should include some reason to believe that those uncertainties can be resolved. If the uncertainties are too great, the review may decide to move forward but plan for further exploration or prototyping early in the project.

The concept is likely complete when it includes all the parts needed to satisfy the system purpose. If some stakeholder need requires $n$ behaviors in the system, the concept has all $n$ behaviors, and all the parts needed to implement each behavior are included.

A good concept is one that is better than other concepts that could potentially meet system purpose, and one that addresses competition from others. “Better” has three aspects. First, one concept is better than another at meeting system purpose if it can meet more stakeholder needs than the other, or has more flexibility to adapt if stakeholder needs change. Second, concepts can be compared on design esthetics that are proxies for desirable system properties like understandability, flexibility for change, or customizability. These esthetics include modularity that encapsulates concerns or the use of well-understood design patterns. Third, a good concept includes little or nothing extra that does not clearly support the system purpose.

A concept that passes its review meets the system’s purpose, meaning that it can satisfy all the stakeholder needs identified in the purpose. As noted earlier (Section 34.6), the initial list of stakeholder needs may or may not be possible, and different approaches might address more or fewer of those needs. By the time the concept gets to review, the system purpose should be revised so that the concept can support all of the needs in the purpose. Satisfying the system purpose at this review should include the stakeholders validating the concept, not just checking that the concept checks all the boxes of the purpose artifacts.

This review, like all reviews, is done by people independent of the main project in order to detect blind spots or biases that the team may have developed. The reviewers can include stakeholders when validation is part of this review; if the validation has already been done, stakeholders do not need to be part of the review team.

The conceptual design review is often a time when a project decides whether to continue with development or to stop work on the system--the go/no go decision. This is a point when the project evaluates the feasibility of building the system and the cost-benefit tradeoff in the work. If the uncertainty in building the system or the cost is too high, then the team decides not to invest more time and resources on the project and turns to some other work.

RFP-driven projects. In an RFP-driven project, where the team develops a proposal for a customer in order to get approval and resources for development, a proposal development phase follows concept development. The conceptual design review determines whether the team has a concept that is good enough for them to proceed on to writing the proposal.

The concept review is also the point at which a team decides whether to pursue completing and submitting a proposal. This decision depends on:

Whether the team has the resources to pursue the work,
Whether the team has a reasonable chance of winning a contract, and
Whether building the system would be worthwile.

Determining whether the team has resources requires estimating the resources needed. For the first steps of purpose and concept development, this may be small, perhaps one or a handful of people to gather information and to get an initial understanding of what the customer wants. As the work progresses, more resources will be needed—to gather more information, to do concept development, to gather competitive market data. At each step of the process, it will become clearer how many people or other resources are needed for the next step of developing the proposal. At the same time, the team must be able to estimate how much resource will be needed to build the system if they win a contract. This will be unknown to start, but as the system concept and architecture work move forward the estimates will improve. The team must develop the architecture enough to be able to determine prices to charge the customer and to be able to determine if the team will have the capacity to do the work. These analyses grow out of the concept and later architecture documents.

Determining whether the team has a reasonable chance of winning is a combination of knowing how the customer will judge proposals, how strong other teams are likely to be, and how well this team can satisfy the customer. This information is gathered in the customer definition document, the competition document, and in how the concept and architecture respond to the customer’s objectives.

Finally, determining whether building the system can be worthwhile depends on the needs of the organization and funder. Does the organization require a particular profit margin? Is there a minimum or maximum contract price that is considered “interesting”? Does the system fit within the organization’s business strategy? These kinds of questions are captured in the objectives of the organization or funder, and analyses use these objectives, concept, and architecture documents to develop an answer to them.

Developing proposals is a complex specialty, and much has been written about it. We refer the reader to ! Unknown link ref for further reading.

34.9 Changing the concept

Changing the concept can be an expensive and error-prone task, but it must be done sometimes either to reflect changes in system purpose (Section 33.10) or to respond to feedback as the team develops the system (Section 34.5.7). A change can be expensive because the concept sets the pattern for the structure of all the components that make up the system. One change to the concept can ripple through many parts of the system, potentially affecting a great many components. On the other hand, if purpose has actually changed or a problem has been found, the concept does need to be changed and the system design adjusted to match.

The team must follow a careful process to make changes to the concept after it has been baselined. Some amount of work will have been done on specification and design; it may have progressed even into implementation. The team needs to trace out the effects of the concept change on all of the specification or design work that has been done, and then adjust those accordingly.This can result in some components being dropped and other added. It can change the functional specifications of other components. It the change happens late in the project, what parts of the system ahve been implemented and verified, the changes may propagate all the way to updated software, hardware designs, and test cases. If the team misses some part of the system, parts of the system might not be consistent with each other and not work correctly together.

In a couple projects, I watched people get confused as the target customer needs and concept changed. People did not catch when some important system concept changed and they continued to design and build to a concept that no longer applied. This disconnect led in turn to hard-to-find system flaws: people designed accurately to the wrong objective, and sometimes it was a long time before someone caught the mismatch. This led to error-prone redesigns of parts of the systems and extra cost.

The implication of this change is that changing the system concept is both a technical matter and a team communication matter. A well-functioning team communicates clearly when the concept changes and provides safety net mechanisms to catch when someone has missed a concept change.

Changes to the concept should begin as tentative works in progress, distinct from any baselined version of the concept. To baseline a concept change, the new version must be reviewed to ensure that it meets the review criteria identified above: feasibility, completeness, compliance. If the changes are extensive, the change review can require nearly as much effort as the original concept review. If the changes are confined to just part of the concept, the review can often be limited to only those parts of the concept affected by the change. Nonetheless, the updated concept should pass the review before being baselined.

The initial concept only needs to be likely feasible, complete, or compliant. As the project development continues, the bar of likelihood raises; by the end of the project, the concept must be fully feasible, complete, and compliant.

Not every request to change the concept will be granted. As with changing the system’s purpose (Section 33.10), the team should evaluate the effects of a concept change before committing to it. Indeed, a change to purpose and change to concept may go together: someone might propose a change to the stakeholder needs the system will address, and as part of evaluating whether to make that purpose change the team develops an update to the system concept in order to estimate how costly the change will be. The changes to purpose and concept may then be accepted and baselined together. On the other hand, if the updated concept shows that the change is not reasonable, then both the changes to purpose and concept may be rejected together. Because of this, it is important to keep tentative, work-in-progress versions of the concept separate from baselined versions.

34.10 Using the concept

34.10.1 Concepts in a visionary project

I defined a visionary project in Section 33.3 as one that does not yet have a specific defined customer or market segment.

Ideally, a visionary project works out potential market segments (sets of potential customers), and investigates those to determine likely wants and needs. The system concept is designed to satisfy this likely system purpose.

Customers often want a great many things that can’t reasonably be satisfied in one system, at reasonable time and cost. Developing a concept is an opportunity to see which needs can be satisfied, and at what cost. Two concepts can be compared by how many customers each concept might satisfy well enough, and thus roughly which approach will be more popular. Concepts can be evolved to reach a cost-market size tradeoff that the project considers best.

A system concept that is aimed at a hypothetical customer is likely to need to change as the project learns more about their customers and narrows down their needs. As noted earlier, the process of deciding to commit to a new concept must be handled carefully so that the entire team works in concert with the changes.

I have joined some projects that decided to begin design and implementation before working out the concept. In one case they did this in order to satisfy potential funders by showing some kind of progress; in another case, most of the people involved were specialists who wanted to get working right away on their part of the system. In both cases the team designed and started building components that did not meet customer needs as those get identified. Both teams established a culture of working based on narrow, incomplete understanding and they only identified and fixed systemic problems with unhappy team upheaval and significant cost. They both had trouble meeting actual customer needs because they did not connect the work on individual components to the system structure and purpose.

34.10.2 Concepts in an RFP-driven project

An RFP-driven project is one where a potential customer issues a request for proposals, and the team develops a proposal in response. The customer request lists their needs and hence defines the system purpose, though the team often will work with the customer to clarify the request.

The project responds to the request with a proposal. The proposal defines what the team will produce, show that it meets the customer needs, and estimates the cost and time involved. The customer uses these proposals to choose which team will get a contract to design and implement the system.

The proposal includes the system concept: the concept defines the system that the customer will get, at the high level.

Because the proposal also includes cost and time estimates, the team usually has to go further than the concept itself and include some high-level design. This design improves the basis for estimating cost and time. Those estimates need greater certainty than what can be worked out from the concept alone.

The team also decides at some point whether to proceed to develop the proposal, and whether to submit a proposal. These decisions depend on whether the team has a viable concept or not, whether the team has the skills and resources to build to that concept, whether the concept will meet customer needs, and whether the project will meet other stakeholder needs, such as profitability. These go/no-go decisions use analyses done on the system concept.

34.10.3 Incremental growth and descope options

A system concept does not have to be all or nothing. A project can choose to design and build a system in phases, starting from something simple and adding capabilities over time.

As noted in Section 33.6, the stakeholders may have asked for more than is achievable with the time or funding available. Concept development is the time when the team does the work that can reveal that the stakeholders are asking for something too complex, including what the complicated parts are and why they will be hard to build.

One way to handle this is to plan to build the system in multiple steps. Choosing a concept that grows over time is a good idea when there is value in getting something in front of a customer quickly and getting their feedback. Planning to grow over time can also address the need to satisfy potential funders who want to see progress and market acceptance before committing additional investment.

A project can also do the opposite: define a system concept, and plan potential ways to descope or remove capabilities if the work proceeds more slowly or takes more resources than expected.

It is worth doing this planning early, rather than sometime later in the project when there is an emergency rush to fix a problem.

Finding options for growing or shrinking a project over time depends on three things: importance, viability, and dependencies among features. Importance involves ranking which parts of the purpose (needs) are least important, and that can be removed from the plan while leaving the most remaining value. Viability is whether the system will be useful with those parts removed.

In the example cubesat system, we had a request to include a capability to control spacecraft attitude to manage atmospheric drag, especially near perigee (the lowest point in the orbit, when atmospheric drag is greatest). This would have been used to try to manipulate how far apart the spacecraft drifted, and to extend or shorten their lifetime on orbit. This capability was not essential to the basic communication and science operations of in the mission, and it was removed from the concept and negotiated out of the system scope. In other words, this feature was of low importance and did not affect viability.

Teams often plan a sequence of things that can be removed or added, reducing or adding to the system scope and purpose step by step. I have developed a set of systems engineering tools over the years to help me build systems following the models in this book. That system started as something simple that could manage tables of requirements (Chapter 37). As time went by I have added capabilities one by one to model the component breakdown (Chapter 41), various ways of specifying component behavior and interaction (Section 42.3), and the information exchanged during interactions (Section 42.4). At each step the system has become more useful, but I didn’t wait until everything was perfectly worked out before starting to use it.

Reduce the system capabilities too much, however, and it isn’t useful to a customer. If there are multiple customers—such as for a visionary project that is aiming for some market segment—reducing the system features can reduce the number of customers who will find the system viable. This can lead to an initial version being viable for only a few customers, with the number of customers growing as more features are added in over time. This can be useful, as it gives a team time to build a solid system foundation and validating customer satisfaction before investing in more features.

Finally, many features are dependent on each other.

In one spacecraft system I worked on, project leadership wanted to launch a minimal spacecraft and add to its software capabilities over time. The team was far behind schedule developing their system, and they were looking for any ways they could get more development time while still meeting the launch schedule they had already paid for. The spacecraft would launch with a basic software image that could operate the spacecraft and check it out on orbit, and the team would then upload new science operation or data analysis capabilities over time. This would give the team more time to get the software written and tested before it was needed, and it would allow for adding new capabilities that weren’t imagined when the mission started.

The difficulty they encountered was that incrementally adding new software capabilities depends on some baseline functions: the ability to communicate well enough to send up new software, and the ability to install and run the new functions. The ability to upload new software implies a communication capability that can move significant amounts of data from ground to the spacecraft. Such communication implies a moderately high bandwidth transceiver, antennas to match, the ability to point the antennas at ground stations, and the ability to schedule ground communication. If the team did not build these in from before launch, it will be nearly impossible to add them later. In other words, adding all the new functions while on orbit depended on significant capabilities being available from the time of launch.

34.10.4 Relation to other lifecycle patterns

The concept development work described in this chapter includes steps from both the NASA Pre-Phase A and Phase A project phases. Pre-Phase A includes developing the concept of operations, while Phase A includes developing the mission architecture. These phases of the NASA lifecycle include many other steps, such as developing various management plans or identifying stakeholders and mission needs.