Making systems—6: Other topics
XV   Other topics
Chapter 70   Standards

70.1 Introduction

Standards are agreements that everyone will implement something the same way, or work with a topic in the same way. They provide norms that allow people to communicate in common language. They also define norms that allow systems or components to interact correctly.

Standards are the product of human effort, with all the attendant opportunities for mistakes. There are many ways that a standard can go wrong, and a good one takes care and attention. The process of making a standard is as much a political process as a technical one.

A standard is a kind of specification (Chapter 36). As a specification, it defines norms. Use of the standard involves conformance to those normative specifications. Much has been written about how requirements must be unambiguous and verifiable, and the same holds for standards: if two people interpret a normative statement in different ways, the standard is not effective.

A useful standard must be generally accessible to those who will use it. Its utility depends on it being commonly understood, which implies that people know about it and can access it. In turn this requires a long-term ability to obtain a copy of the standard, typically from a source that is known to provide a reliable copy. These archives must be trusted to maintain the integrity of the copies they store and provide.

This has led to standards organizations that coordinate standards development and provide a reliable way to obtain copies. Some standards are managed by international, regional, or national quasi-governmental organizations. Some of the standards managed by these organizations are referenced in regulation and thus carry legal weight. Other standards are industry-specific, coordinating activities such as networking protocols. Some industry-specific standards are developed by ad hoc groups in order to promote some shared technology. There are also standards internal to organizations that apply only to people working there.

70.2 What are standards for?

Standards are fundamentally about enabling multiple separate teams to work together in some way. A standard provides a common language, a common design model to inform understanding, or common objectives.

Within a system, a standard specifies behaviors and functions at a boundary components. It enables interoperability by defining interfaces (including behaviors of endpoints). If two components conform to a standard, they should be able to communicate properly with each other. A standard also enables interchangeability by defining the boundary a component must conform to. If two components implement the same standard interface, one can be substituted for the other.

Standards create the possibility for multiple independent teams to design and build things that work together. This differs from a specification within a system, which is limited to that one project.

Standards also create opportunity by allowing new designs and implementations for a component that can be added to existing systems. Different organizations can create different implementations, leading to competition that incentivizes improvement and quality. Users can choose the implementation that best suits their needs, taking advantage of different designs that make different design tradeoffs.

A standard differs from a specification within one system project in another way. A standard is not the definition of one thing; it is the definition of constraints that many different things must follow. In building one system, a component’s specification defines how that one component will behave; it most often leads to one design and one implementation. A standard, on the other hand, is about enabling the design and development of multiple independent components. This leads to two properties of standards:

70.2.1 Kinds of standards

There are as many kinds of standards as there are subjects for which there can be a norm. Some of the most common include:

This list is not exhaustive, and some standards fit into more than one category.

70.3 Key principles for a good standard

Many standards that are proposed and published do not end up useful. The following principles help a standard avoid this fate.

A purpose. A standard that gets used has a clear reason to be. A standard is like any other product: it must address a need that people have in order for people to be willing to invest the effort in learning the standard and using it.

Standard development projects, therefore, must develop the same kind of purpose that any system development project does. That purpose should collect and record who that standard’s stakeholders are and what each one will expect of the results. These expectations define requirements that the standard must meet, and every standard is judged by how well it meets those requirements. Standards development efforts that take the time to document these requirements often have an advantage because a potential user can see whether the expectations addressed in developing the standard meet the needs that the user has. Such an effort also has advantages when a standards development organization evaluates a proposed standard to determine if it should be adopted and published.

A defined scope or subject. An effective standard has a well-defined scope: it is about something specific. Someone considering standards that may apply to them can understand how the standard applies to what they are building, if it does.

Some standards provide specifications for multiple related subjects. This can be a problem for two reasons. First, it makes it harder for someone building a system to determine whether a standard applies to their needs. Second, it can result in a standard that addresses each subject poorly compared to standards focused on each subject individually.

At the same time, some situations call for standards for a number of related subjects. For example, consider the many standards involved in Internet email. There are several standards for transmitting messages between nodes, as well as many standards related to message content, security, the underlying networking infrastructure, and so on. Each of these standards focuses on one subject, and the collection of standards is designed to accomplish the overall goal of providing email services.

Safety and security first. A good standard puts correct function first, ahead of other considerations. For example, safety and security takes priority over matters like regulatory compliance. If safety or security must be compromised to comply with regulation or to meet a business need, then the standard should not be published.

Even more so, business needs must not take precedence over safety or security. Too often organizations cause public harm by putting business objectives ahead of safety. I have several times encountered companies that have developed a product and then promoted parts of that product’s design as a standard. In most cases there have been essential safety or correctness flaws in the standards. In every case the standard has not been useful for many potential users—precisely because the point of the business putting the standard forward is to ensure that their product gets used in preference to other products. See the section on extensibility below for more.

Independent use. A standard is only a standard if people independent of those who wrote the standard can use the standard to create an implementation that interoperates or interchanges with other implementations of the same standard.

A standard is thus a mechanism for communicating between those who develop the standard and those who try to use it. Just recording facts is not sufficient; the standard must include explanation of how the various facts in the standard are related. The standard must bring the reader’s attention to points that are easy to miss or that can be misunderstood. The standard-writer must accurately imagine the knowledge that the standard reader will have, and possibly make clear what background information the reader must have in order to properly understand the facts. In other words, it is just as important how the information is organized and how the standard’s text guides the user to understanding the material as it is to record the specific facts.

In some recent efforts I have heard complaints from the team developing a standard that some things like an index would be a problem because it would introduce redundancy in the document. My response is that such guides are a necessary part of writing a standard. While it is true that it increases the effort required by those who develop the standard—the parts must be kept consistent with each other—this is simply part of the task. Standards-writing is not easy.

Re-use of intellectual effort. Part of the value of a standard is that it packages up a set of design decisions that others can use without having to recreate that design. This value is separate from and in addition to value that may come from interoperation or interchange.

This is particularly important for designs that require significant intellectual effort to get right. For example, developing cryptographic algorithms involves specialist knowledge that most people do not have. Most people cannot do the analysis to show that such an algorithm has desired properties. By choosing some particular algorithm that has been designed and analyzed, its user can have confidence that the algorithm they use will meet their requirements.

The implication of this is that standards that are expected to provide properties that most people will not be able to show need to be accompanied by the analyses that show that they will correctly provide those properties. The analysis might be incorporated as an appendix to the standard, or be published separately and be referenced in the standard, but either way the analysis must be available to the standard’s users.

Extensibility. A standard, once adopted and used, is extraordinarily costly to change. It is commonly understood that it is more expensive to change a component’s implementation than it is to change its specification before building the implementation. The cost of changing a standard is far higher, because a successful standard may have led to many independent implementations that must be updated if the standard changes.

This situation implies that if a standard needs to change, it is likely to be difficult to impossible to do so if complying with an updated standard requires changing the design of implementations. If someone identifies new potential uses for the standard, but those uses change something in the standard’s functions, it will often be difficult to convince everyone else to change their implementations to accommodate the new one. If someone finds a better solution to some problem that the standard addresses, it may not be possible to get everyone to make the investment to change to the new, better solution.

I worked on one standard some years ago that was defining interfaces for accessing an intelligent storage device. People had general agreement on the basic functions that the interface should provide, but not for functions like how to report and correct data errors on the device. One group proposed functions for reporting when some data were corrupted, identifying which data items had problems, and methods for fixing that corruption. That proposal was included in the standard. Unfortunately, the functions that were included did not meet the needs of many of the potential users.

By prematurely defining those data error functions, the standard locked out many potential uses. More seriously, this prevented later defining a better solution. Had the standard remained silent on these functions and allowed different projects to develop potential solutions, the experience of many groups could have been combined into a solution that worked for most users.

In the end that standard failed. The interface never gained market acceptance and no products ever came to market that used it.

The lesson is if a feature is included in a standard and adopted by its users, that feature is locked in place for a very long time. If that feature is mature and generally useful, that’s a good thing. If the feature is not mature, improving or correcting the standard is sometimes expensive and more often impossible.

It is better to explicitly omit from a standard features that cannot be argued with certainty that they are mature enough and generally applicable enough not to require change.

If there is a feature that must be included but that is likely to change, standards can be designed for extensibility. Protocols for encrypted communication take this approach, for example: they explicitly provide for supporting multiple encryption algorithms, giving each one an identifier and supporting negotiation between the communication endpoints to determine which algorithms to use. In this way the structure protocol standard can remain unchanged while new algorithms are added incrementally, with backward compatibility for those who do not need the new algorithms yet.

70.4 Developing a standard

Developing a standard is a process, like developing other things. A standard has a purpose; there is a concept for what it covers; and it contains specifications of how that concept can be expressed in compliance.

The process differs from other systems-building projects mainly in two ways. It is almost always developed by multiple people in a community, with different motivations and needs, rather than by people within one organization. Its end product is typically a document that will be used by people who have no relationship with those who developed the standard, and who cannot ask for clarification.

70.4.1 Standards development work flow

The overall work flow for developing a standard is similar to developing a system or its components: it starts with the purpose for the standard, proceeds through developing a concept for the standard’s content and requirements for the standard, proceeding on to writing the standard itself.

Most standards are developed for an existing standards development organization (SDO). These organizations provide formal status to a standard—indeed, regulation often refers to standards published by such organizations. The organization defines processes for creating and approving a standard, and provides logistical assistance. The organization provides the archival and distribution mechanisms for standards once published, and advertises them to potential users.

Some standards are written as independent efforts, with their weight as a standard determined by projects that voluntarily use the standard.

undisplayed image

As with all development, the actual flow of activity does not likely follow the tidy, linear sequence in the diagram. Instead, people will make progress on different tasks in parallel. They will find gaps in stakeholder understanding while working on the standard’s requirements. They will get input from external parties during writing that causes a re-thinking of the concept. This is normal and necessary. By the time that the standard document is published, the results should appear as if the results from one task inform the subsequent tasks, and that the collection of artifacts are all consistent with each other.

Many standards are revised some time after being published, as people get experience using it and find issues, or as the needs the standard addresses change. When this happens, the work flow is reiterated.

70.4.2 Differences from system development

While the overall flow of developing a standard is similar to the flow for developing a system, there are significant differences.

Standards efforts are often started after multiple organizations have developed solutions for similar problems, and there is a desire to unify the solutions. This means, first, that there will be multiple alternative concepts at the very start of the work, rather than starting purely from an analysis of stakeholder needs and proceeding to develop a concept and design from scratch. Second, there will be multiple stakeholders that are likely to have competing interests to see their solution become the standard.

Standard development is therefore a more political task than most ordinary system development. Where most well-functioning system projects have a leadership with authority to make design decisions, standard development involve many parties, all of which can potentially veto a decision. This places greater emphasis on listening to multiple viewpoints and on consensus-building than in system development.

The product of the development is typically a document, rather than an operable system. The document must be understandable and accessible to those who will use the standard to design and build things. Those people may be doing so long after the standard was developed, with no recourse to those who wrote it years before. This leads to a need for a clear, consistent writing style and careful attention to explanations that can often be abbreviated in ordinary system development.

Most standards are developed under the sponsorship of a standards development organization. These organizations require their projects to follow a defined development process, in order to ensure the quality of the result and general agreement on its contents. The organizations also have standards for the language used, the layout of the document, and formats and tools used.

70.4.3 Standard development as a collective action problem

A standard has value when it gets used widely; adoption is thus key to its value.

A standard constrains the set of acceptable solutions for its subject. This limits the ways that one organization can differentiate its product from products developed by other organizations, which in turn creates incentives for some organizations not to adopt the standard or to try to bend the standard toward their preferred approach.

At the same time, standards also create incentives for some organizations to cooperate. When a standard allows two different products from two different organizations to work together, then both organizations can benefit.

In other words, standard development is a collective action problem. The standard’s benefits accrue when all, or almost all, potential users do so, while many potential users will have a private incentive not to use it. In some cases this tension is resolved when the standard is of such universal value that almost everyone sees the benefit. In other cases the potential users must be coerced, sometimes by regulation and more often by market forces demanding interoperable products.

Working on a standard requires at least some participants to be aware of these conflicting incentives and address them in the work.

70.4.4 Development as a community process

Because a standard only has value when multiple parties adopt it, a standard must be attractive to a large fraction of the community of potential users.

This leads to development processes that emphasize broad consultation and regular feedback. Most standards development organizations have mechanisms to involve as many people as possible in the drafting and review process. They also typically have carefully-defined processes for reviewing and approving a draft standard, requiring either consensus or near-consensus on a draft before it is accepted for publication.

Note that this does not imply that every standard requires universal acceptance. A standard only needs to be adopted by enough projects to get the advantage of interoperability and independent development.

People who develop standards outside such organizations need to find ways to ensure sufficiently broad appeal of those standards.

Successful standard development projects require deliberate effort to work well in such an open, community environment.

It is helpful, as part of developing the initial standard purpose, to work out who might be affected by a standard or who might use it. This information can guide how people reach out to others to get their inputs or to get them involved in the development.

A smooth standard development effort also creates clear opportunities for everyone to participate. I have found that regular discussion meetings, occasional face-to-face discussions, and availability for electronic discussion both as groups and privately all help people express their concerns and work toward agreement.

Consensus-oriented processes can devolve into bickering and disagreement unless managed. The participants developing a standard usually do not all have the same incentives: one party wants to encode something they have already built as the standard, while another has specific features that will benefit them but not others. Different parties come to different understandings of the needs the standard must meet. The standard development team needs leadership that can help all the parties air these differences and come to some compromise solution. The team also needs leadership that can keep the work focused on one definition of the work and avoid adding other subjects to the standard.

70.4.5 Verification

As a standard is developed, it should be measured against the objectives it set out to address. This ensures that the result actually meets its stakeholders’ needs, and that everything in the standard is driven by those needs.

The verification is usually handled by reviews, both informally during development and more formally during the review and acceptance phase. Because the standard is largely text, as opposed to an operable component, traditional testing approaches aren’t helpful.

The team must have actually defined and recorded those objectives if it is to perform meaningful verification. This starts in the purpose development phase, working out who the stakeholders are and what their needs are, and then is refined when developing specific requirements that the standard must meet.

Obtaining consensus on what these objectives are is essential to obtaining agreement on the final standard. Without agreement on objectives, each participant will bring their own agenda to the standard and it will end up reflecting an incoherent set of purposes.

In practice many published standards are not clear on their objectives. They only lay out the technical specification for their subject without defining the ideas that explain why the content is what it is. This creates two later problems. First, when someone evaluates a standard that they might use, they have to reverse-engineer what the standard’s purpose is. Second, when it comes time to update the standard, the team making changes has to try to reverse-engineer the purposes that the original used in order to determine how the updated purpose changes the content.

70.4.6 Validation

A standard is known to “work” when someone uses it to develop a new, independent implementation that is conformant with the standard and interoperable with other implementations.

Doing so requires:

There is no one mechanism to validate the standard in this way except by trying to use it. However, there are methods by which one can reduce the frequency of problems. First, one can test whether the standard’s content is clearly explained using outside reviewers—asking them to read the standard and note ambiguities they find, and then having them explain the standard back to the team that wrote it to check that they were able to extract the correct meanings from the content. Second, teams can build implementations and test that the implementations work as required and can interoperate. Finally, experience helps: knowing what didn’t work in previous efforts can help the standard developers avoid ambiguities in the first place.

Having a set of conformance verification methods helps both in validating the standard initially and in developing implementations later. Many standards provide a set of verification tests, either as an official, normative part of the standard or as an unofficial adjunct to it.

70.4.7 Revision

Virtually every standard gets revised at some point. It may be revised because new uses have been found for the subject, meaning that the needs the standard addresses have changed. It may be revised because people have found errors or ambiguities in the published version. It may also be revised as improved solutions are discovered for the problems the standard addresses.

The process to revise a standard is essentially the same as writing the original standard, except that the original material already exists. The process begins by determining who the stakeholders are now, and what needs should now be addressed that are not in the existing version. Working out a concept for the changes to make follows, and so on.

Working out changes is easier and more accurate if the team that developed the existing version has left documentation of their work: the purpose, the rationales for decisions, the subtle choices reflected in the design, or analyses used to check the existing version (Section 8.2.5). Without this information, the team developing a revision will have to reverse engineer the information, which always gets some of the information wrong.

Revisions to a standard must deal with questions of compatibility. Whether existing implementations of the old standard should interoperate with implementations of the new standard depends on the needs of stakeholders; there is no one right answer. If a revision is correcting small errors or adding functionality, then backward compatibility is usually easy, especially if the original standard was designed to support extensibility (as discussed in an earlier section). If the changes are such that interoperability is not desired—perhaps because the revision is correcting some serious flaw in the original—then ensuring that implementations to the old and new standards visibly do not interoperate is often desirable. For example, a revision could change the shape of a hardware connector or include standard version information in message exchanges. Silently appearing to function is to be avoided when there are incompatible changes.

70.5 Content of a standard

A standard defines a set of specifications that others will follow to build systems or perform activities sometime later. These specifications are, therefore, the primary content of the standard.

The specifications are not enough in themselves. They need context in order to be understood properly, and people will evaluate the scope of the standard before choosing to use it.

Most standards organizations have a common structure for the standards they publish. This often starts with an introduction defining the scope of the standard and its relationship to other standards. Most standards include boilerplate such as lists of references, acronyms, a glossary, the authors, revision history, and the terms under which the standard is published. The rest of the document contains the text that defines the requirements, along with explanations and analysis.

The document’s content can be divided into two parts: the normative and the non-normative. The normative part consists of what a compliant implementation or usage must follow. The non-normative part contains the explanations and context, but not the requirements. It is important that the text makes it clear which material is which. The non-normative, explanatory portion is often written in colloquial language that explains but is not precise. Some standards organizations define specific markers in the text to highlight the requirements, such as labeling a paragraph as a requirement or putting the word “SHALL” or “MUST” in capital letters in a requirement sentence.

The normative content, which expresses requirements, expresses concepts in deontic logic [McNamara22]. Deontic logic is concerned with expressions of obligation, prohibition, and permissibility. The specifications for a system and its components usually use only a limited form of this logic, focused on what is required or prohibited (See Chapters 36 and 37). Most standards use a wider range of these expressions. A standard can express things that may be included or may be omitted. For example, a standard might say that if some capability A is implemented, there are requirements B, C, and D for how it must function. Such a statement does not require an implementation to include A. It does, however, prevent a compliant implementation from including A but doing so in a different way. The IETF standards include the following text that defines the kinds of specifications that their standards can include:

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [BCP14]…

The non-normative content of a standard provides explanation, rationale, and justification.

Explanations provide information that helps the readers understand correctly what the standard is saying. Some of that is background about the problems the standard is addressing; that is, the context for the specification.

More important, though, are the explanations that add to the specifications themselves. Succinct and precise specification statements are hard to write unambiguously. They often rely on a specific meaning of a term or assume some fact is already understood. Without explaining these meanings and facts the reader has difficulty guessing correctly what the specification statement means. In one standard I worked on, a specification statement used the term “person”. This statement used the term in its legal sense, which is different from common usage. In common usage, a person is a human being or similar, while in legal terms a person is someone or something that can take on legal responsibilities, such as executing a contract. People reading the statement were confused until we added an explanation that the text used the word in its legal sense.

Other explanatory material guides the reader to understand subtle points that might be covered by specification statements but that are not obvious until they have explored all the implications and corner cases in the specifications. It is easy for a casual reader to miss these implications, so if they are important they should be called out in the explanations.

The rationale is a record of why choices were made in the standard. It is useful to reviewers, who will be checking those choices. It is important for people who later revises the standard, so that they understand the choices that they may need to change. The rationale does not necessarily need to be part of the standard document itself, as long as it remains available to people who need it.

The justifications are a record of why and how the standard, as written, is correct. It shows how the published standard meets its objectives. It also includes analyses of key principles, such as safety or security, that provide evidence of meeting those objectives. In one standard, we had a set of safety objectives. While developing the standard, we analyzed the design to see if it would be safe in those ways. Where we found it was not, we modified the standard to fix the problems. At the end, we had an argument for why someone should believe that the published standard met the safety objectives. Again, the justifications do not necessarily need to be part of the published document as long as they are maintained and accessible when needed. Some standards, however, include at least outlines of the justification arguments in appendixes.

70.6 Distribution and archival

70.7 Using a standard