Tools are things that people use while designing and building the system. The tools are not part of the system itself; they are not delivered to an end user. Their purpose is to help the team do their job. Each project will have its own needs for tools, so this list is meant to inspire ideas rather than prescribe what may be needed for building any specific system. There are, however, some common principles for selecting and managing tools.
This chapter brings together information about many different kinds of tools, with references to the other parts of this volume that discuss details.
Please note: I do not recommend specific tools.
There are a few general principles that apply to selecting tools generally.
First, most tools will be used for shared work. Tools should be evaluated on how well they help the team work together. Computer-based tools that manipulate shared data, such as CAD tools, should make it easy for multiple people to access the information concurrently. They should support the project’s approach to versioning information Section 17.4.3. Physical tools should be accessible to those who need to use them. This is especially important to consider if people work in multiple physical locations.
Second, many tools require training to be used effectively and safely. The project must ensure that each person has been trained to use a tool safely before they are allowed to use it. That implies that tools should be evaluated on the quality of educational material available on how to use them.
Third, good tools are integrated so that they work together. Tools that can share information can provide greater value to the team than ones that cannot.
Next, tools should support the general life cycle and procedures the project uses. They should fit into the project’s procedures for managing artifacts, versioning them, and reviewing them.
Finally, tools should be secure. Good tools will support the project’s overall approach to security, including controlling access to information based on a person’s role in the project. This includes both electronic and physical security.
This section provides an overview of all the kinds of tools discussed elsewhere in this volume, with references to the sections that provide details. The overview can serve as a checklist for a team working out what tools they need.
The tools for storing and managing artifacts are discussed in Section 17.4.4.
Electronic artifacts. Alternatives include:
Hardware artifacts. These can use:
As I will discuss in Part VIII, the team will develop specifications for system components. A specification defines a component’s external interfaces—in systems terms, how the component is part of functional and non-functional relationships (Section 12.2).
There are several kinds of specifications (Section 36.5), including requirements, interface definitions, and models.
Requirements (Chapter 37). Requirements provide textual statements of things that are to be true about a system or component. Requirements can be managed using:
I list a number of considerations for selecting requirements management tools in Section 37.13.
Interface definitions (Chapter 39). Interface definitions specify how one component can interact with others. These can be written using:
Models (Chapter 38). Mechanical, mathematical, electronic, behavioral, and other kinds of models are used as specifications. Relevant tools include:
A project’s design phase works out a set of designs for the system and its components that satisfy the corresponding specifications (Part IX).
A design records the structure of each component—whether a high-level, composite component or a low-level component (Chapter 11). It also records analyses that lead to designs and rationales for how a design ended up as it did.
There are two kinds of design artifacts: the breakdown structure and the designs themselves. The model in Section 11.4 has six parts to a component design: form, state, actions or behaviors, interfaces, non-functional properties, and environment.
Breakdown structure (Chapter 41). I recommend that the component designs be organized by the component breakdown structure. This structure organizes the components into a hierarchical name space, giving each one a unique identifier and showing how one component is made out of others.
On most projects, I have used a spreadsheet to list all the components, the breakdown organization, and their names. This has worked well enough, and I am not aware of tools that explicitly support such organization.
Form (Section 42.1). The form represents the aspects of a component that do not change, or only very slowly. The design of physical components is generally handled using CAD tools. These tools use notations or drawing standards appropriate to each subject.
State, actions, behaviors (Sections 42.2 and 42.3). This part of a design addresses the parts of a component that change readily.
Non-functional properties (Section 42.5). These properties change slowly and are not part of the component’s form.
Environment (Section 42.6). This is the environment in which the component is expected to operate, or in which it may be stored. This is usually recorded in text.
These tools help the design process by providing feedback on how well a particular design works. They also are used when verifying a proposed design.
These tools help translate designs or implementations into operable components that can be integrated into a running system, or used for testing.
The built artifacts will need to be stored and tracked, as discussed above .
Physical artifacts. The building of physical artifacts is, in effect, manufacturing one or a small number of those artifacts. These can be built in multiple ways.
In-house building will require maintaining a stock of the materials used in the components. This may include a stockroom of pre-acquired parts, such as metal or plastic stock and fasteners, or suppliers that can provide the needed material quickly.
The building process should be deterministic: if the team builds multiple instances of the same component, the components should all look and behave the same way. This places constraints on whatever tools and procedures are used to build the components.
Software artifacts. Software artifacts are built by translating source code into binary and packaging it in forms that can be installed on a target system.
The software build process must be repeatable: if the same software is built twice, the result should be identical in behavior (differing only in things like version numbers, timestamps, or affected signatures). This usually means that the software build tools should be under configuration management so that identical tools will be used each time.
Testing involves taking a component, or collection of components, and subjecting it to some sequence of activities to verify that the component behaves as specified.
Testing occurs at two different times during system development: as people are building parts of the system and when a component or the system is being verified for final acceptance. These two uses lead to somewhat different needs in the tools for testing.
Tests need to be accurately reproducible: someone should be able to run a test one time on one component, then run the same test later on the same component and get the same result. Of course some component behaviors are not fully deterministic, but accounting for that, one should be able to count on passing a test meaning that the component really does meet the specification being tested. If a test fails, people need to be able to reproduce what happened to understand the flaw and to determine whether a fix works.
Reproducibility places constraints on testing tools. Physical tests will need to be done in consistent environments, using control and measurement tools that can be calibrated to ensure they are behaving consistently. Software tests similarly need to be run in controlled environments.
Hardware testing. Testing hardware components can range from measurements of single components to integration tests of subsystems or even the complete system. The tools available vary widely, depending on the kind of testing being done.
All hardware testing will involve:
Tools that support testing electronic components can include:
Tools for testing mechanical components include:
Integrated system testing can go well beyond the tools listed here. Flight testing a new aircraft, for example, is far more complex than suggested by these tools. I leave the design and operation of such testing to others better versed in it.
Software testing. Software testing generally involves setting up a number of test cases or scenarios, running the software being tested, and recording the results. There are many different tools that can be used, and these depend on the kind of test being performed and the environment or language being used.
Categories of tools include:
The team uses other kinds of information to manage its operations—about the team, about procedures, plans, and to support decision-making.
Team information (Section 17.3.2). This information is organized around the roster of who is on the team, along with their roles and authority.
This information links to other other tools, some of which are often outside a project’s scope. These include:
These relationships get updated whenever someone joins the team, leaves the team, or their role changes. Using tools that guide people through the procedures for these updates will make the changes more accurate.
Life cycle and procedures (Chapter 20). Teams follow a project life cycle and procedures to do their work. These consist of steps that people should follow to get specific tasks done.
Workflow management tools exist to help guide people through these procedures. These tools can help by:
Plans and tasking (Section 20.6 and Section 20.7). The project maintains plans for how the system-building work will move forward and the work currently in progress. The plan records the work that the project will be doing, at varying levels of confidence and detail, while tasking tracks the specific work that people have been assigned. This information is used both to make sure that the team do the work that is needed, without important tasks getting forgotten, and for forecasting the time and resources needed to move forward.
Maintaining plans and tasking is an exercise in managing a lot of detail. Many tools are available to help with these.
In practice, many of the tools available have been designed for projects other than systems-building, and do not support systems projects well. Many project scheduling tools are based on methodologies worked out for predictable work like building construction, where the tasks can be known fairly accurately in advance. These tools often are organized around a Gantt chart of the work, prompting their users to estimate duration and make task assignments early in the project. This works poorly in systems projects that have significant uncertainty early in the project, and where the degree of certainty (or predictability) improves unevenly as time goes by. This often results in a false sense of confidence in the project’s schedule early on, and requires a lot of effort to try to keep the schedule adapted as work moves forward.
It is worth spending effort working out how a project will manage its planning and tasking, and ensure that any tools chosen will support that approach.
Support. Project operations maintains other kinds of information as well, for which tools are sometimes available. These include:
Good tools can enhance a team’s performance. Poorly chosen or implemented tools can harm it. One must choose tools carefully and apply thought to how they are implemented and used.
A project’s tools are themselves systems, and should be treated with the same care as the system being built for a customer.
Each tool should have a purpose. Spending the time to work out who will benefit from a particular tool, both directly and indirectly, can provide useful guidance when choosing between options for that tool.
The engineering support tool industry has generated many products that can be used, meaning there are often many possibilities to choose from. While sometimes the team can cut a decision process short because they already have experience with one particular tool, in the other cases it is worth setting out some criteria for making the choice.
Factors that can influence the choice of tool include:
Once a tool has been chosen, it will need to be purchased or built, and deployed for the team to use. This usually requires finding space for the tool, whether that is physical space in a lab or capacity on a compute server. The acquired tool will need to be deployed and integrated into the project’s systems: adding information about the tool to an inventory database, setting up a service schedule if needed, integrating software systems with the project’s security mechanisms.
Team members will need to learn how to use new tools. For some tools, this can amount to providing a written introduction or presentation on how the tool works. More complex tools will require more formal training. If there are safety or security risks in using the tool, the project should ensure that people are required to receive training before using the tool. It is common to track formally which people have gotten this kind of safety training.