Safety Software Guide for Use with 10 CFR 830 Subpart A, Quality Assurance Requirements, and DOE O 414.1C, Quality Assurance

DISTRIBUTION: INITIATED BY:

http://www.directives.doe.gov Office of Environment, Safety and Health

DOE G 414.1-4

Approved 6-17-05

Certified 11-3-10

SAFETY SOFTWARE GUIDE

for USE with

10 CFR 830 Subpart A, Quality Assurance

Requirements, and DOE O 414.1C, Quality Assurance

[This Guide describes suggested nonmandatory approaches for meeting requirements.

Guides are not requirements documents and are not construed as requirements in any

audit or appraisal for compliance with the parent Policy, Order, Notice, or Manual.]

U.S. DEPARTMENT OF ENERGY

Washington, D.C.

NOT MEASUREMENT

SENSITIVE

DOE G 414.1-4 i (and ii)

6-17-05

FOREWORD

This Department of Energy (DOE) Guide is approved by the Office of Environment, Safety and

Health and is available for use by all DOE and National Nuclear Security Administration

(NNSA) elements and their contractors. This Guide revises and supersedes earlier guidance

identified in Appendix B to include new and updated information.

Comments, including recommendations for additions, modifications, or deletions, and other

pertinent information, should be sent to the following.

Gustave E. Danielson, Jr. Richard H. Lagdon, Jr. (Chip)

U.S. DOE U.S DOE

Office of Quality Assurance Programs Director, Office of Quality Assurance Programs

1000 Independence Avenue SW 1000 Independence Avenue SW

EH-31/270CC EH-31/270CC

Washington, DC 20585-0270 Washington, DC 20585-0270

Phone: 301-903-2954 Phone: 301-903-4218

Fax: 301-903-4120 Fax: 301-903-4120

E-mail: [email protected] E-mail: [email protected]

Guides are part of the DOE directives system and are used to provide supplemental information

regarding DOE/NNSA expectations for fulfilling requirements contained in Policies, Rules,

Orders, Manuals, Notices, and Regulatory Standards. Guides are also used to identify

Government and non-Government standards and acceptable methods for implementing

DOE/NNSA requirements. Guides are not substitutes for requirements nor do they introduce new

requirements or replace technical standards used to describe established practices and

procedures.

DOE G 414.1-4 iii

6-17-05

CONTENTS

BACKGROUND .............................................................................................................................v

1. INTRODUCTION ...............................................................................................................1

1.1 Purpose.....................................................................................................................1

1.2 Scope........................................................................................................................1

1.3 Responsibility for Safety Software..........................................................................3

1.4 Safety Software Quality Assurance .........................................................................3

1.5 Software Quality Assurance Program......................................................................3

2. SAFETY SOFTWARE TYPES AND GRADING..............................................................5

2.1 Software Types ........................................................................................................5

2.2 Graded Application..................................................................................................6

3. GENERAL INFORMATION..............................................................................................8

3.1 System Quality and Safety Software .......................................................................8

3.2 Risk and Safety Software.........................................................................................9

3.3 Special-Purpose Software Applications.................................................................10

3.3.1 Toolbox and Toolbox-Equivalent Software Applications........................10

3.3.2 Existing Safety Software Applications.....................................................11

3.4 Continuous Improvement, Measurement, and Metrics..........................................11

3.5 Use of National/International Standards................................................................12

4. RECOMMENDED PROCESS..........................................................................................13

5. GUIDANCE.......................................................................................................................13

5.1 Software Safety Design Methods...........................................................................13

5.2 Software Work Activities ......................................................................................17

5.2.1 Software Project Management and Quality Planning.............................17

5.2.2 Software Risk Management.....................................................................19

5.2.3 Software Configuration Management .....................................................21

5.2.4 Procurement and Supplier Management.................................................22

5.2.5 Software Requirements Identification and Management ........................23

5.2.6 Software Design and Implementation .....................................................24

5.2.7 Software Safety........................................................................................25

5.2.8 Verification and Validation.....................................................................27

iv DOE G 414.1-4

6-17-05

CONTENTS (continued)

5.2.9 Problem Reporting and Corrective Action..............................................30

5.2.10 Training Personnel in the Design, Development, Use, and

Evaluation of Safety Software .................................................................30

6. ASSESSMENT AND OVERSIGHT.................................................................................31

6.1 General...................................................................................................................31

6.2 DOE and Contractor Assessment...........................................................................32

6.3 DOE Independent Oversight..................................................................................32

APPENDIX A. ACRONYMS AND DEFINTIONS.............................................................. A-1

APPENDIX B. PROCEDURE FOR ADDING OR REVISING SOFTWARE TO

OR DELETING SOFTWARE FROM THE DOE SAFETY

SOFTWARE CENTRAL REGISTRY..........................................................B-1

APPENDIX C. USE OF ASME NQA-1-2000 AND SUPPORTING STANDARDS

FOR COMPLIANCE WITH DOE 10 CFR 830 SUBPART A

AND DOE O 414.1C AND SAFETY SOFTWARE.....................................C-1

APPENDIX D. QUALITY ASSURANCE STANDARDS FOR SAFETY

SOFTWARE IN DEPARTMENT OF ENERGY NUCLEAR

FACILITIES ................................................................................................. D-1

APPENDIX E. SAFETY SOFTWARE ANALYSIS AND MANAGEMENT

PROCESS ......................................................................................................E-1

APPENDIX F. DOE O 414.1C CRITERIA REVIEW AND APPROACH

DOCUMENT.................................................................................................F-1

APPENDIX G. REFERENCES ............................................................................................. G-1

DOE G 414.1-4 v (and vi)

6-17-05

BACKGROUND

The use of digital computers and programmable electronic logic systems has increased

significantly since 1995, and their use is evident in safety applications at nuclear facilities across

the Department of Energy (DOE or Department) complex. The commercial industry has

increased attention to quality assurance of safety software to ensure that safety systems and

structures are properly designed and operate correctly. Recent DOE experience with safety

software has led to increased attention to the safety-related decision making process, the quality

of the software used to design or develop safety-related controls, and the proficiency of

personnel using the safety software.

The Department has recognized the need to establish rigorous and effective requirements for the

application of quality assurance (QA) programs to safety software. In evaluating Defense

Nuclear Facilities Safety Board (DNFSB) recommendation 2002-1 and through assessing the

current state of safety software, the Department concluded that an integrated and effective

Software Quality Assurance (SQA) infrastructure must be in place throughout the Department’s

nuclear facilities. This is being accomplished through the Implementation Plan for Defense

Nuclear Facilities Safety Board Recommendation 2002-1, Quality Assurance for Safety Software

at Department of Energy Defense Nuclear Facilities.

To ensure the quality and integrity of safety software, DOE directives are being developed and

revised based on existing SQA industry or Federal agency standards This resulted in the

development and issuance of DOE O 414.1C, Quality Assurance, dated 6-17-05, which includes

specific SQA requirements, this Guide and the DOE Standard 1172-2003, Safety Software

Quality Assurance Functional Area Qualification Standard, dated December 2003. The SQA

requirements are to be implemented by DOE and its contractors. Nuclear facility contractors

must implement the SQA requirements under their QA program for 10 CFR 830, Subpart A,

Quality Assurance Requirements. Thus, the intent of this Guide is to provide instructional

guidance for application of DOE O 414.1C safety software requirements.

DOE G 414.1-4 1

6-17-05

1. INTRODUCTION

1.1 PURPOSE

This Department of Energy (DOE or Department) Guide provides information plus acceptable

methods for implementing the safety software quality assurance (SQA) requirements of DOE

O 414.1C, Quality Assurance, dated 6-17-05. DOE O 414.1C requirements supplement the

quality assurance program (QAP) requirements of Title 10 Code of Federal Regulations

(CFR) 830, Subpart A, Quality Assurance, for DOE nuclear facilities and activities. The safety

SQA requirements for DOE, including the National Nuclear Security Administration (NNSA),

and its contractors are necessary to implement effective quality assurance (QA) processes and

achieve safe nuclear facility operations.

DOE promulgated the safety software requirements and this guidance to control or eliminate the

hazards and associated postulated accidents posed by nuclear operations, including radiological

operations. Safety software failures or unintended output can lead to unexpected system or

equipment failures and undue risks to the DOE/NNSA mission, the environment, the public, and

the workers. Thus DOE G 414.1-4 has been developed to provide guidance on establishing and

implementing effective QA processes tied specifically to nuclear facility safety software

applications. DOE also has guidance

for the overarching QA program, which includes safety

software within its scope. This Guide includes software application practices covered by

appropriate national and international consensus standards and various processes currently in use

at DOE facilities.

This guidance is also considered to be of sufficient rigor and depth to ensure

acceptable reliability of safety software at DOE nuclear facilities.

This guidance should be used by organizations to help determine and support the steps necessary

to address possible design or functional implementation deficiencies that might exist and to

reduce operational hazards-related risks to an acceptable level. Attributes such as the facility

life-cycle stage and the hazardous nature of each facility’s operations should be considered when

using this Guide. Alternative methods to those described in this Guide may be used provided

they result in compliance with the requirements of 10 CFR 830 Subpart A and DOE O 414.1C.

Another objective of this guidance is to encourage robust software quality methods to enable the

development of high quality safety applications.

1.2 SCOPE

This Guide is intended for use by all DOE/NNSA organizations and their contractors to assist in

developing site and facility specific safety SQA processes and procedures compliant with

10 CFR 830 Subpart A and DOE O 414.1C.

The Department’s objectives for safety software requirements include—

DOE G 414.1-2, Quality Assurance Management System Guide for use with 10 CFR 830.120 and DOE O 414.1,

dated 6-17-99.

See Appendix G for list of consensus standards.

2 DOE G 414.1-4

6-17-05

• grading SQA requirements based on risk, safety, facility life-cycle, complexity, and

project quality requirements;

• applying SQA requirements to software life-cycle phases;

• developing procurement controls for acquisition of computer software and hardware that

are provided with supplier-developed software and/or firmware;

• documenting and tracking customer requirements;

• managing software configuration throughout the life-cycle phases;

• performing verification and validation (V&V)

processes;

• performing reviews of software configuration items, including reviewing the safety

implications identified in the failure analysis and fault tolerance design; and

• training personnel who use and apply software in safety applications.

The scope of this Guide includes software applications that meet safety software definitions as

stated in DOE O 414.1C. This includes software applications important to safety that may be

included or associated with structures, systems, or components (SSCs) for less than hazard

category 3 facilities. Safety Software includes safety system software, safety and hazard analysis

software and design software, and safety management and administrative control software.

Safety system software is software for a nuclear facility

that performs a safety function as part of

an SSC and is cited in either (1) a DOE approved documented safety analysis or (2) an approved

hazard analysis per DOE P 450.4 Safety Management System Policy, dated 10-15-96, and the

DEAR clause.

Safety and hazard analysis software and design software is software that is used to classify,

design, or analyze nuclear facilities. This software is not part of an SSC but helps to ensure the

proper accident or hazards analysis of nuclear facilities or an SSC that performs a safety

function.

Safety management and administrative controls software is software that performs a hazard

control function in support of nuclear facility or radiological safety management programs or

Technical Safety Requirements or other software that performs a control function necessary to

provide adequate protection from nuclear facility or radiological hazards. This software supports

eliminating, limiting, or mitigating nuclear hazards to workers, the public, or the environment as

addressed in 10 CFR 830, 10 CFR 835, and the DEAR ISMS clause.

Additional definitions are included in Appendix A, Acronyms and Definitions.

Verification and validation in this Guide includes ASME’s NQA-1 terms design verification and acceptance

testing.

Per 10 CFR 830, quality assurance requirements apply to all DOE nuclear facilities including radiological facilities

(see 10 CFR 830, DOE Std 1120, and the DEAR clause).

DOE G 414.1-4 3

6-17-05

Although this Guide has been developed for DOE nuclear facility software, it may also be useful

for ensuring the quality of other software important to mission critical functions, environmental

protection, health and safety protection, safeguards and security, emergency management, or

assets protection.

1.3 RESPONSIBILITY FOR SAFETY SOFTWARE

The Assistant Secretary for Environment, Safety and Health has the lead responsibility for

promulgating requirements and guidance through the directives system for safety software per

DOE O 414.1C. The organizations that use software should determine whether to qualify the

software for safety applications. Organizations should coordinate SQA procedures with their

respective Chief Information Officers and other appropriate organizations. DOE line

organizations are responsible for providing direction and oversight of the contractor

implementation of SQA requirements.

1.4 SAFETY SOFTWARE QUALITY ASSURANCE

The scope of the Department’s QA Rule, 10 CFR 830 Subpart A, is stated as “This subpart

establishes quality assurance requirements for contractors conducting activities, including

providing items or services, that affect, or may affect, nuclear safety of DOE nuclear facilities.”

The scope of the QA Rule encompasses the contractor’s conduct of activities as they relate to

safety software (items or services). Therefore the contractor’s QAP includes safety software

within its scope. DOE O 414.1C establishes the safety software QA requirements to be

implemented under the Rule. 10 CFR 830 Subpart A and DOE O 414.1C require contractors to

perform safety software work in accordance with the applicable criteria.

The various sections of this Guide discuss the application of the QA criteria from DOE O 414.1C

and 10 CFR 830 Subpart A to the ten SQA work activities. Table 1 provides an illustration of

how the SQA work activities satisfy the QA criteria.

1.5 SOFTWARE QUALITY ASSURANCE PROGRAM

It is important that SQA is part of an overall QAP required for nuclear facilities in accordance

with 10 CFR 830 Subpart A and DOE O 414.1C. Regardless of the application of the software,

an appropriate level of quality infrastructure should be established and a commitment made to

maintain this infrastructure for the safety software.

An SQA program establishes the appropriate safety software life-cycle practices, including

safety design concepts, to ensure that software functions reliably and correctly performs the

intended work specified for that safety software. In other words, SQA’s role is to minimize or

prevent the failure of safety software and any undesired consequences of that failure. The rigor

imposed by SQA is driven by the intended use of the safety software. More importantly, the rigor

of SQA should address the risk of use of such software. Effective safety software quality is one

method for avoiding, minimizing, or mitigating the risk associated with the use of the software.

4 DOE G 414.1-4

6-17-05

Table 1. An Illustration of Quality Assurance (QA) Criteria (10 CFR 830 Subpart A & DOE O 414.1C)

Applicability to Software Quality Assurance (SQA) Work Activities

SQA Work

Activities

10 CFR 830

QA Criteria and

DOE O 414.1C

Software (sw) project

management and

quality planning

Sw risk mgmt

Sw configuration

mgmt

Procurement &

supplier mgmt

Sw rqmts

identification &

mgmt

Sw design &

implementation

Sw safety

Verification &

validation

Prblm rpting &

corrective action

Training of … safety

Program X X X X X X X X X X

Training & Qualific ation X

Quality Improvement X X

Documents and Records X X X X X X X X X X

Work Processes X X X X X X X X X X

Design X X X X

Procure ment X

Inspection & Acceptance

Testing

Management Assessment X X X X X X X X X

Independent Assessment X X X X X X X X X X

Note: This table is only an illustration of QA criteria applicability. Actual application will be described in the organization’s QA program

and safety software work process documents. For example, an independent assessment may be performed on any safety software quality

element.

DOE G 414.1-4 5

6-17-05

The goal of SQA for safety system software is to apply the appropriate quality practices to

ensure the software performs its intended function and to mitigate the risk of failure of safety

systems to acceptable and manageable levels. SQA practices are defined in national and

international consensus standards. SQA cannot address the risks created by the failure of other

system components (hardware, data, human process, power system failures) but can address the

software “reaction” to effects caused by these types of failures. SQA should not be isolated from

system level QA and other system level activities. In many instances, hardware fail-safe methods

are implemented to mitigate risk of safety software failure. Additionally other interfaces such as

hardware and human interfaces with safety software should implement QA activities.

2. SAFETY SOFTWARE TYPES AND GRADING

2.1 SOFTWARE TYPES

Software typically is either custom developed or acquired software. Further characterizing these

two basic types aids in the selection of the applicable practices and approaches for the SQA work

activities. For the purposes of this Guide, five types of software have been identified as

commonly used in DOE applications: (1) custom developed, (2) configurable, (3) acquired,

(4) utility calculation, and (5) commercial design and analysis.

Developed and acquired software types as discussed in American Society of Mechanical

Engineers (ASME) NQA-1-2000, Quality Assurance Requirements for Nuclear Facility

Applications are compatible with these five software types. Developed software as described in

ASME NQA-1-2000 is directly associated with custom developed, configurable, and utility

calculation software. Acquired software included in this Guide is easily mapped to that of

acquired software in ASME NQA-1-2000. ASME NQA-1-2000 uses acquired and procured

software terms interchangeably.

This Guide includes an additional software type of commercial

design and analysis software that is not directly related to either developed or acquired software.

Safety software quality requirements can only be specified through work activities described in

contractual agreements with the supplier of the facility design and analysis services.

Custom developed software is built specifically for a DOE application or to support the same

function for a related government organization. It may be developed by DOE or one of its

management and operating (M&O) contractors or contracted with a qualified software company

through the procurement process. Examples of custom developed software includes material

inventory and tracking database applications, accident consequence applications, control system

applications, and embedded custom developed software that controls a hardware device.

Configurable software is commercially available software or firmware that allows the user to

modify the structure and functioning of the software in a limited way to suit user needs. An

example is software associated with PLCs.

ASME NQA-1-2000, Quality Assurance Requirements for Nuclear Facility Applications, Subpart 2.7 Section 300,

American Society of Mechanical Engineers, New York, New York, 2001, p. 105.

6 DOE G 414.1-4

6-17-05

Acquired software is generally supplied through basic procurements, two-party agreements, or

other contractual arrangements. Acquired software includes commercial off-the-shelf (COTS)

software, such as operating systems, database management systems, compilers, software

development tools, and commercial calculational software and spreadsheet tools (e.g., Mathsoft’s

MathCad and Microsoft’s Excel). Downloadable software that is available at no cost to the user

(referred to as freeware) is also considered acquired software. Firmware is acquired software.

Firmware is usually provided by a hardware supplier through the procurement process and

cannot be modified after receipt.

Utility calculation software typically uses COTS spreadsheet applications as a foundation and

user developed algorithms or data structures to create simple software products. The utility

calculation software within the scope of this document is used frequently to perform calculations

associated with the design of an SSC. Utility software that is used with high frequency may be

labeled as custom software and may justify the same safety SQA work activities as custom

developed software.

With utility calculation software, it is important to recognize the difference

between QA of the algorithms, macros, and logic that perform the calculations versus QA of the

COTS software itself. Utility calculation software includes the associated data sets, configuration

information, and test cases for validation and/or calibration.

Commercial design and analysis software is used in conjunction with design and analysis

services provided to DOE from a commercial contractor. An example would be where DOE or

an M&O contractor contracts for specified design services support. The design service provider

uses its independently developed or acquired software without DOE involvement or support.

DOE then receives a completed design. Procurement contracts can be enhanced to require that

the software used in the design or analysis services meet the requirements in DOE O 414.1C.

2.2 GRADED APPLICATION

Proper implementation of DOE O 414.1C will be enhanced by grading safety software based on

its application. Safety software grading levels should be described in terms of safety

consequence and regulatory compliance. This Guide utilizes the grading levels and the software

types (custom developed, configurable, acquired, utility calculations, and commercial design and

analysis tools) to recommend how the SQA work activities are applied. The grading levels are

defined as follows.

Level A: This grading level includes safety software applications that meet one or more of the

following criteria.

1. Software failure that could compromise a limiting condition for operation.

2. Software failure that could cause a reduction in the safety margin for a safety SSC that is

cited in DOE approved documented safety analysis.

3. Software failure that could cause a reduction in the safety margin for other systems such

as toxic or chemical protection systems that are cited in either (a) a DOE approved

ASME NQA-1-2000, op.cit., Part 4, Subpart 4.1, Section 101.1, p. 227.

DOE G 414.1-4 7

6-17-05

documented safety analysis or (b) an approved hazard analysis per DOE P 450.1 and the

DEAR ISMS clause.

4. Software failure that could result in nonconservative safety analysis, design, or

misclassification of facilities or SSCs.

Level B: This grading level includes safety software applications that do not meet Level A

criteria but meet one or more of the following criteria.

1. Safety management databases used to aid in decision making whose failure could impact

safety SSC operation.

2. Software failure that could result in incorrect analysis, design, monitoring, alarming, or

recording of hazardous exposures to workers or the public.

3. Software failure that could comprise the defense in depth capability for the nuclear

facility.

Level C: This grading level includes software applications that do not meet Level B criteria but

meet one or more of the following criteria.

1. Software failure that could cause a potential violation of regulatory permitting

requirements.

2. Software failure that could affect environment, safety, health monitoring or alarming

systems.

3. Software failure that could affect the safe operation of an SSC.

The grading level criteria should provide for a higher grade level for software in nuclear facilities

categorized as Category 1, 2 or 3 and the lower grading level for software in facilities

categorized as less than Category 3. Table 2 illustrates the association of grading criteria

described above to facility categorization.

Using the grading levels and the safety software types in Table 2, select and implement

applicable software quality work activities from the following list to ensure that safety software

performs its intended functions. DOE O 414.1C specifies that ASME NQA-1-2000, Quality

Assurance Requirements for Nuclear Facility Applications, or other national or international

consensus standards that provide an equivalent level of quality assurance requirements as ASME

NQA-1-2000 must be used. As specified in DOE O 414.1C, the standards used must be specified

by the user and approved by DOE. This Guide provides acceptable implementation strategies and

appropriate standards for these work activities.

1. Software project management and quality planning.

2. Software risk management.

3. Software configuration management (SCM).

8 DOE G 414.1-4

6-17-05

4. Procurement and supplier management.

5. Software requirements identification and management.

6. Software design and implementation.

7. Software safety.

8. V&V.

9. Problem reporting and corrective action.

10. Training of personnel in the design, development, use, and evaluation of safety software.

Table 2. Grading Criteria and Facility Categorization Illustration

Nuclear Facilities

1,2 3

Nuclear Facilities

Safety Software A B C A B C

Safety System Software

X X X

Safety & Hazard Analysis

Software & Design

Software

X X X X X X

Safety Management &

Admin Controls Software

X X X X

Safety and hazard analysis software and design software includes software used to classify facilities. Because this

software is used before the facility classification determination, the safety and hazard analysis software and design

software type has been identified as being applicable for all grading levels in all categories of facilities.

The determination of what constitutes safety software is made by the organization applying the

software based upon the requirements in DOE O 414.1C, and 10 CFR 830 Subparts A and B.

The application of the software determines whether it is safety related and how it should be

graded. Therefore, the organization applying the software is responsible for evaluating and

designating the software as safety software and then ensuring that the software development and

operations have followed the appropriate QA procedures.

3. GENERAL INFORMATION

3.1 SYSTEM QUALITY AND SAFETY SOFTWARE

Maintaining the integrity, safety, and security of all DOE assets and resources is paramount for

DOE’s mission. Since software is an integral part of DOE’s resources, the integrity, safety, and

security attributes of its software resources are critical to DOE’s mission. All three attributes are

interdependent since compromising the security access could obviously present a potential safety

hazard. If the integrity of either the data or application itself has been compromised either

DOE G 414.1-4 9

6-17-05

accidentally or maliciously, safety could be compromised. Therefore when safety software is

being addressed, the integrity and security issues should likewise be addressed.

Other system level issues impacting safety software are the availability of trained and

knowledgeable personnel to develop, maintain, and use the software; human factor issues such as

understandability of the displays or ambient lighting conditions; and potential electromagnetic

interference/radio frequency interference, which should be analyzed. Fault tolerance and

common cause failure issues, performance requirements, and proper identification and analysis

of functional requirements that have safety, security or integrity implications need to be

propagated to the safety software.

From the foregoing, it can be seen that there are several interdependencies and tradeoffs that

should be addressed when integrating software into safety systems. The necessity for robust

software quality engineering processes is obvious when safety software applications are required.

However, just ensuring that a “good” software engineering process or that V&V activities exist

is not sufficient alone to produce safe and reliable software.

The life-cycle process should focus

upon the safety issues in addition to the basic software quality engineering principles. Both of

these concepts are detailed in this Guide.

3.2 RISK AND SAFETY SOFTWARE

Software rarely functions as an independent entity. Software is typically a component of a

system much in the same way hardware, data, and procedures are system components. Therefore,

to understand the risk associated with the use of software, the software function should be

considered a part of an overall system function.

The consequences of software faults need to be addressed in terms of the contribution of a

software failure to an overall system failure. Issues such as security, training of operational

personnel, electromagnetic interference, human-factors, or system reliability have the potential

to be safety issues. For example, if the security of the system can be compromised, then the

safety software can also be compromised. Controlling access to the system is key to maintaining

the integrity of the safety software. Likewise, if human factor issues such as ambient lighting

conditions and ease of use for understandability are important, the risks need to be addressed

either via design or training. For PLCs or network safety software applications, electromagnetic

interference could offer potential risks for operation of the safety software system.

Once the software’s function within the overall system’s function is known, the appropriate

software life-cycle and system life-cycle practices can be identified to minimize the risk of

software failure on the system. Rigor can then be applied commensurate with the risk. Managing

the risk appropriately is the key to managing a safety software system. Unless risks and

trade-offs of either doing or not doing an activity are evaluated, there is not a true understanding

of the issues involved regarding the safety software system. Obviously, time and resource

constraints should be balanced with the probability of occurrence and the potential consequences

versus an occurrence of the worst case scenario. If the safety systems staff zealously and

Leveson, Nancy, Safeware: System Safety and Computers, Addison Wesley, 1995.

10 DOE G 414.1-4

6-17-05

religiously inappropriately invokes the strictest rigor for a Level B application, then the

application has the potential never to get fielded properly. On the other hand, if the process

activities are only minimally or inappropriately performed for a Level A software safety

application, then very adverse consequences could potentially occur for which no mitigation

strategy exists. Appropriate project management is a risk management strategy and especially

so for safety software applications.

3.3 SPECIAL-PURPOSE SOFTWARE APPLICATIONS

Several categories of software have a unique purpose in safety-related functions required to

support DOE nuclear facility operations. This section contains an overview of the

special-purpose software and the additional considerations that should be addressed by SQA

programs, processes, and procedures.

3.3.1

Toolbox and Toolbox-Equivalent Software Applications

Toolbox codes represent a small number of standard computer models or codes supporting DOE

safety analysis. These codes have widespread use and are of appropriate qualification for use

within DOE. The toolbox codes are acknowledged as part of DOE’s Safety Software Central

Registry. These codes are verified and validated and constitute a “safe harbor” methodology.

That is to say, the analysts using these codes do not need to present additional defense as to their

qualification provided that the analysts are sufficiently qualified to use the codes and the input

parameters are valid. These codes may also include commercial or proprietary design codes

where DOE considers additional SQA controls are appropriate for repetitive use in safety

applications and there is a benefit to maintain centralized control of the codes. The following six

widely applied safety analysis computer codes have been designated as “toolbox” codes.

• ALOHA (chemical dispersion analysis)

• CFAST (fire analysis)

• EPIcode (chemical dispersion analysis)

• GENII (radiological dispersion analysis)

• MACCS2 (radiological dispersion analysis)

• MELCOR (leak path factor analysis)

The current designated “toolbox” codes and any software recognized in the future as meeting the

“toolbox” equivalency criteria are no different from other custom developed safety software as

defined in Section 2.1. Consequently, software of this category should be developed or acquired,

maintained, and controlled applying sound software practices as described in Section 5 of this

Guide.

In the future, new versions of software may be added to the Central Registry while the older

versions are removed. Over time, some of the software may be retired and recommended not to

be used in DOE safety analysis. Still other software may be added through the formal

toolbox-equivalent process, having been recognized as meeting the equivalency criteria. Thus,

the Central Registry collection of safety software applications will be expected to evolve as

DOE G 414.1-4 11

6-17-05

dditional information on the detailed toolbox SQA procedures, criteria, and evaluation plan; the

entral Registry

software life-cycle phases, usage, and application requirements change. Appendix B address

the process for adding new software applications and versions to, and removal of retired softwa

from, the Central Registry.

evaluation of the software relative to current SQA criteria (i.e., assessment of the margin of the

deficiencies or “gap” analysis); user guidance documentation; description of the

toolbox-equivalent process; and code-specific information may be found in the C

portion of the DOE SQA Knowledge Portal (http://www.eh.doe.gov/sqa/central_registry.htm).

3.3.2

Existing Safety Software Applications

E g proved under a QA program consistent with

xisting safety software should be identified and controlled prior to evaluation using the graded

est

3.4 CONTINUOUS IMPROVEMENT, MEASUREMENT, AND METRICS

Lord Kelvin stated “If you can not measure it, you can not improve it.” This truism especially

OE O 414.1C criterion 3, Quality Improvement, specifies that processes should be established

xistin software that has not been previously ap

DOE O 414.1C and has been identified as safety software should be evaluated using the graded

approach framework that is described in Section 5. This software is often referred to as legacy

software. In many cases this category of software originally met DOE or industry requirements

but SQA for existing software was not updated as the SQA standards were revised.

approach framework in this Guide. The evaluation performed should be adequate to address the

correct operation of the safety software in the environment it is being used. This evaluation

should include (1) identification of the capabilities and limitations for intended use, (2) any t

plans and test cases required demonstrating those capabilities, and (3) instructions for use within

the limitations.

One example of this evaluation is a posteriori review

as described in American

Nuclear Society (ANS) standard, ANS 10.4. Future modifications to existing safety software

should meet all safety software work activities in DOE O 414.1C associated with the changes

the safety software.

applies to safety software systems. Metrics used throughout the life-cycle should bolster the

confidence that the software applications will achieve their mission in a safe and reliable manner.

If design, testing, or software reliability measures are unknown, then there is no assurance that

the safety software has sufficient robustness to minimize the risks.

and implemented to detect and prevent problems. Measurements and the metrics developed from

these measures can be indicators for potential future problems, and thus, steps can be initiated to

prevent the occurrence. For long term avoidance of problems, continuous improvement methods

should be implemented to determine the root causes and eliminate the events that could lead to a

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 302, p. 105.

American National Standards Institute (ANSI)/American Nuclear Society (ANS) 10.4-1987 (R1998), Guidelines

for the Verification and Validation of Scientific and Engineering Computer Programs for the Nuclear Industry,

ANS, 1998, Section 11, pp. 29–32.

Lord Kelvin ( Sir William Thomson, 1824–1907)

12 DOE G 414.1-4

6-17-05

dards to develop and

xperts

ME is the nationally accredited body for the development of nuclear

part A

the requirements generally apply to safety software

other standards useful in achieving compliance

ll the

applications.

reoccurrence. Metrics further provide qualitative or quantitative indicators of the improvements

or lack thereof when a process or work activity has been modified. Metrics are the evidence that

an improvement has occurred. Both Institute of Electrical and Electronic Engineers (IEEE)

Standards 982.1 and 982.2 provide recommendations for what metrics to use and when in th

software life-cycle phase applying the metric is most appropriate.

3.5 USE OF NATIONAL/INTERNATIONAL STANDARDS

Title 10 CFR 830 Subpart A and DOE O 414.1C require the use of stan

implement a QAP. National/international standards facilitate a common software quality

engineering approach to developing or documenting software based upon a consensus of e

in the particular topic areas. Many national and international standards bodies have developed

software standards to ensure that the recognized needs of their industry and users are

satisfactorily met.

In the United States, AS

facility QA standards. DOE O 414.1C cites ASME NQA-1-2000 or other national or

international consensus standards that provide an equivalent level of quality assurance

requirements as ASME NQA-1-2000 as the appropriate standard for QAPs applied to

nuclear-related activities (e.g., safety software). The ten QA criteria in 10 CFR 830 Sub

and DOE O 414.1C are mapped to ASME NQA-1-2000 in Appendix C. DOE O 414.1C also

requires that additional standards be used to address specific work activities conducted under t

QAP, such as safety software.

In the case of ASME NQA-1-2000, Part I,

work activities. For example, Requirements 3, 4, 7, 11, 16, and 17 for Design Control,

Procurement Document Control, Control of Purchased Items and Services, Test Contro

Corrective Action, and Quality Assurance Records (respectively) can have specific safety

software applicability. In addition, ASME NQA-1-2000, Part II, Subpart 2.7, and Part IV,

Subpart 4.1, specifically address “quality assurance requirements for computer software fo

nuclear facility applications” and “guide on quality assurance requirements for software”

(respectively). As stated in the introduction of Part II, Subpart 2.7, this subpart “provides

requirements for the acquisition, development, operation, maintenance, and retirement of

software.” Table 3 provides a cross reference of ASME NQA-1-2000 with the ten SQA wo

activities in DOE O 414.1C. Although ASME NQA-1-2000 provides excellent process guidan

for a software quality engineering process for managing a software development, maintenance,

or procurement or otherwise acquiring software, the detailed guidance for safety software is not

provided within this standard.

Appendix D of this Guide includes references to

with 10 CFR 830 Subpart A and DOE O 414.1C for safety software work activities. It should be

noted that the use of the standards discussed can aid in the development of a robust safety

software quality engineering process and a resulting software product that is adequate for a

safety software applications. Use of consensus standards can promote a robust safety software

quality engineering process and a resulting software product that is adequate for safety software

ASME NQA-1-2000, op.cit., Part I.

DOE G 414.1-4

6-17-05

Recognizing that there are five OE listed in

Section 2.1, the safety software analyst needs a defined process to enable a determination of

fety

5.1 SOFTWARE SAFETY DESIGN M

Safety should be designed into a system, just as quality should be built into the system. Safe

n nt, uses two primary approaches:

(

ces is

the first approach to developing high quality software systems. These practices can be

e more integrated model, Capability Maturity

Leveson, op. cit., p. 398.

4. RECOMMENDED PROCESS

safety software applications types within D

hat needs to be accomplished for each of the respective software safety applications. In

addition, the safety software analyst needs a process to support the integration of software sa

into the system safety process to improve system and software design, development and test

efforts. Lastly, the process to manage each of the five application types should support the

planning and coordination of the software safety tasks based on established priorities. Appendix

E of this Guide presents the details of a risk-based graded approach for the analysis and safety

software management process for (1) custom developed, (2) configurable, (3) acquired,

(4) utility calculations, and (5) commercial design and analysis tools.

5. GUIDANCE

ETHODS

desig of a system, in which safety software is a subcompone

1) applying good engineering practices based upon industry proven methods and (2) guiding

design through the results of hazard analysis. Identifying and assessing the hazards is not enoug

to make a system safe. The information from hazard analysis needs to be factored in the

design.

Applying industry accepted software engineering and software quality engineering practi

generally

pplied to safety software to improve the quality and add a level of assurance that the software

performs its safety functions as intended. DOE O 414.1C requires SQA work activities, referred

to as work activities, to be performed for safety software. Many national and international

consensus standards, such as ASME NQA-1-2000, ANS 10.4, and the IEEE software

engineering series provide detailed guidance for performing the work activities. Section 3.5 of

this Guide describes some of these standards.

Software process capability models such as the Software Engineering Institute’s legacy Softwar

Capability Maturity Model (SW-CMM) and th

odel Integration (CMMI), are proven tools to assist in the selection of practices to perform for

achieving a level of assurance that the processes performed will produce the desired level of

quality for safety software. The CMMI has two approaches: staged and continuous. For

organizations introducing a software process improvement program, these models should be

considered.

14 DOE G 414.1-4

6-17-05

Table 3. ASME NQA-1-2000 Cross Reference to DOE Software Quality Assurance (SQA) Work Activities

SQA Work

Activities

ASME

NQA-1

Software (sw)

project management

& quality planning

Sw risk

management

Sw configuration

management

Procurement &

supplier

management

Sw requirements

identification &

management

Sw design &

implementation

Sw safety

Verification &

validation

Prblm reporting &

corrective action

Training of

…

safety sw

Organization Req. 1,

1A-1,

200

2A-2, 301

Quality

Assurance

Program

SP 2.7,

400

2A-2, 300

1A-1,

200

SP 2.7,

400

2A-2, 301,

502

Req. 1

Req. 2,

100

SP 2.7,

400

SP 4.1,

400

Req. 2

SP 2.7,

402

1A-1, 303 Req.

2A-2

, 600

Design Control SP 2.7,

400

SP 4.1,

101, 200,

404, 406

Req. 3,

802

SP 2.7,

203

SP 4.1,

203

SP 2.7,

300

SP 4.1,

300

Req. 3,

800

SP 2.7,

400

SP 4.1,

400

Req. 3,

800

SP 2.7,

400

SP 4.1,

400

Req. 3,

800

SP 2.7,

402

SP 4.1,

100

Req. 3,

801.4,

801.5

Req. 11,

400

SP 2.7,

402.1, 404

SP 4.1,

402.1, 404

Req. 15

Req. 16

SP 2.7,

204

SP 4.1,

204

Procurement

Document

Control

Req. 4

Instructions,

Procedures, and

Drawings

Req. 5 Req. 5

Document

Control

Req. 6

SP 2.7,

201

SP 4.1,

201

Req. 6

SP 2.7,

201

SP 4.1,

201

DOE G 414.1-4 15

6-17-05

SQA Wo

Activities

SME

NQA-1

…

Software (s

project management

& quality planni

Sw risk

manageme

Sw configuratio

manageme

Procurement &

supplier

manageme

Sw requirements

identification &

manageme

Sw design

implementatio

Sw safety

Verification &

validation

Prblm reporting &

corrective acti

Training of

safety sw

Cont

Purc ms

SP 4.1,

300

rol of

hased Ite

and Services

Req. 7,

P 2.7,

Identific

Control of It

ation and

ems

eq. 3,

802

SP 2.7,

203

SP 4.1,

203

Control of Special

Processes

Inspection

402.1, 404

Req. 3,

801.4,

801.5

Req.

400

SP 2.7,

402.1, 404

SP 4.1,

Test Control Req. 3,

801.4,

801.5

Req. 11,

400

SP 2.7,

402.1, 404

SP 4.1,

402.1, 404

16 DOE G 414.1-4

6-17-05

SQA Work

Activities

ASME

NQA-1

Software (sw)

project management

& quality planning

Sw risk

management

Sw configuration

management

Procurement &

supplier

management

Sw requirements

identification &

management

Sw design &

implementation

Sw safety

Verification &

validation

Prblm reporting &

corrective action

Training of …

safety sw

Control of

Measuring and

Test Equipment

SP 4.1,

101.3

Handling,

Storage, and

Shipping

Inspection, Test

and Operating

Status

Control of

Nonconforming

Req. 15

SP 2.7,

SP 4.1,

Items 2

Corrective Action Req. 16

SP 2.7,

204

SP 4.1,

204

Quality

Assurance

Records

SP 2.7,

201

SP 4.1,

Audits Req. 1 8

DOE G 414.1-4

6-17-05

For safety s

ystems, hazards and accident analyses are performed at the system level and then for

ny subcomponent of the system that potentially could have an adverse effect on safety. Since

oftware is a subcom , hazard analys afety software is

performed. Hazard analysis is best performed periodically throughout the life-cycle of the safety

software development and operations to reassess h s safety of the system and its

software. The information from these hazard an ke design decisions related to

the safety software and its associated safety system.

5.2 SOFTWARE WORK ACTIVITIES

Software should be lled in a traceable, planned, and orderly manner. The safety software

quality work activities defined in this section provide the basis for planning, implementing,

maintaining, and operating safety software. The work activities for safety software include tasks

such as software project planning, SCM, and risk analysis that cross all phases in the life-cycle.

Additionally, the work activities include tasks that are specific to a life-cycle phase. These work

activities cover tasks during the development, maintenance, and operations of safety software.

The work activities should be implemented based upon the graded level of the safety software

and the applicable software type. Table 4 provides a summary of the mapping between software

type, the grading levels, and the ten SQA work activities. Not all work activities will be

applicable for a particular instance of safety software. This Guide indicates where these work

activities may be omitted. However, the best judgment of the software quality engineering and

safety system staffs should take precedence over any optional work activities presented in this

Guide.

5.2.1

Software Project Management and Quality Planning

As with any system, project management and quality planning are key elements to establishing

the foundation to ensure a quality product that meets project goals. For software, project

management starts with the system level project management and quality planning. Software

specific tasks should be identified and either included within the overall system planning or in

separate software planning documents.

These tasks may be documented in a software project management plan (SPMP), an SQA plan

(SQAP), a software development plan (SDP), or similar documents. They also may be embedded

in the overall system Typically the SPMP, SQAP, and/or SDP are the

controlling documents that define and guide the processes necessary to satisfy project

requirements, including the software quality requirements. These plans are initiated early in the

project life-cycle and are maintained throughout the life of the project.

The software project management and quality planning should include identifying all tasks

associated with the sof devel ent and ent of services,

ponent of the system is specific to the s

the

yses

aza

used to m

and

aal

contro

level planning documents.

tware opm procurement, including procurem

ration (CM

ng (CM

Mellon U

Capability Maturity M m ability Matu tegration, Version

1.1, CMMI for Softw , Stag resentatio -2002-TR-029,

ESC-TR-2002-029, C e Eng ng Institu .

odel Integ

are Engineeri

arnegie

MI) Pro

MI-S

niversity

duct Tea

W, V1.1)

, Softwar

, Cap

ed Rep

ineeri

rity Mod

n, CMU

te, 2002

el In

/SEI

18 DOE G 414.1-4

6-17-05

Table 4. Mapping Safety Software Types and Grading Levels to Software Quality Assurance (SQA) Work Activities

SQA Work

Activity

Level

Custom

Developed

Configurable

Acquired

Utility Calcs

Commercial

D & A

Custom

Developed

Configurable

Acquired

Utility Calcs

Commercial

D & A

Custom

Developed

Configurable

Acquired

Utility Calcs

Commercial

D & A

Software (Sw)

project

management &

quality planning

Full Full Grade Grade n/a Full Full Grade Grade n/a Grade Grade Grade Grade n/a

Sw risk

management

Full Full Full Full n/a Grade Grade Grade Grade n/a Grade Grade Grade Grade n/a

Sw configuration

management

Full Grade Grade Grade Grade Full Grade Grade Grade Grade Grade Grade Grade Grade Grade

Procurement &

supplier

management

Full Full Full Full Full Full Full Full Full Full Full Full Full Full Full

Sw requirements

identification &

management

Full Full Full Full Full Full Full Full Full Full Full Full Full Full Full

Sw design &

implementation

Full Grade n/a Grade n/a Full Grade n/a Grade n/a Full Grade n/a Grade n/a

Sw safety

Full Full Full n/a n/a Grade Grade Grade n/a n/a Grade Grade Grade n/a n/a

Verification &

Validation

Full Full Full Grade n/a Grade Grade Grade Grade n/a Grade Grade Grade Grade n/a

Problem

reporting &

corrective action

Full Full Full Grade Full Full Full Full Grade Full Full Grade Grade Grade Grade

Training of …

safety Sw

Full Full Full Full n/a Grade Grade Grade Grade n/a Grade Grade Grade Grade n/a

DOE G 414.1-4 19

6-17-05

estimate of the duration of the tasks, resources allocated to the task, and any dependencies. The

planning sh n of the tasks and y rele t i

A-1-20 nsensus nda s

planning documents that are

d resources to ass in the id fic n and

curem

quality and tware de op nt ning ntifies and th ftwa hases

grad QA and software developme ctivit pe d during software

elopment or maintenance. The software quality and softwa ng ng ivitie d rigor

lem e dependent on the identified ading el ty software and the

ity of DOE or its contractors to build quality in and assess the quality of the safety software.

ause SQ are ove basic quality and ftware gi

vities, suc risk ma , p lem r rting co e ons, V&V,

uding software reviews and testing, may be further detailed in separate plans. These plans

ac ed in th pl will be discussed la id

tware project management and quality planning fully apply custo ev ped a

figura pes for b L l A d Lev saf o F evel nd

B acquired and utility calculation and all Level C software applications, software project

ent and qua plannin sk n rade his grading i de th

catio g of all if t ware tasks. W i s e so re

-cycle may include little or n ftware development activities, the software project and

pl st likely be part of the overall system el o ility nning.

s work not apply co er l design and analysis software because the

ject man quality ni ac ties as ciated th

lysis softw med by the service supplier. DOE controls the SQA activities of that

ware through procurement agreements and specifications.

So anage t

tware risk manag nt provides a disciplined environmen r proa e ision

tinuou t can go n ete ine w risks im t t dres d

ent actions to address those risks.

Because risk management is such a fundamental tool

t managem an gr ar softw proje a en lthou

mes associated with safety analysis of tentia ilures ftw k nagement

ses on the risks to the successful c pletion of the software project. The risks addressed by

work ject ma em ri associated with the successful completion of a

ware ap hereas Section 5.2.7 addresses the risks associated with the potential

modes of the software.

ould include a descriptio

00, several

an van nformation. In addition to

goo

pro

Sof

and

dev

of i

abil

Bec

acti

incl

and

Sof

con

Lev

ide

life

qua

Thi

pro

ana

soft

5.2

Sof

con

imp

for

som

focu

this

soft

fail

co sta rd

,15

provide details of

ist enti atio and description of the software development

ent tasks.

tware

any

sof

e S

vel me plan ide

nt a

gui

rfor

des

e so re p

ing of th ies

re e

lev

ine

eri

safe

act s an

mp entation will b gr

AP and S

h as SC

rall

nagem

epo

and

nee

rre

ring plans, some quality

ctivent rob acti and

the tivities identifi ese ans ter in this Gu e.

ety s

m d

re.

elo

or L

A able software ty oth eve an el B ftwa

nagem

ntifi

lity

ckin

g ta

sign

s ca

ican

be g

soft

d. T sh

nsta

ould

nce

nclu

of th

ftwan and tra here

o so

lity anning will

activity do

lev project r fac pla

mm cia

agement a

are are

perfor

plan ng tivi so wi commercial design and

ftware Risk M men

eme

t fo

are

ctiv

rtan

dec

o ad

making to

s, ansly assess w wro g, d rm hat po

lem

projec

eti

ent, it is inte al p t of

are

l fa

ct m

, so

nag

e ris

t. A

ent activity are pro nag sks

plication w

ure

nd E ctronic E

EE, 1998.

EE Standard f

eference Doc

ers Software Q

stitu

anagem

EE Std

QAS21.0

nergy Q

te of Electrical a le n St So

ent Plans, IE

730-2002, IE

1.00-1999 (

E uality Mana

gine

or Soft

ument)

uality Assu

ers (IEE

ware Q

, Softw

E), d 1058

uality Ass

are Risk M

nce Subco

-1998,

urance Pla

anagemen

mmittee, d

IEEE S

ns, IEE

t: A Pr

ated 2-

tand

E, 20

acti

200

ard for

02.

cal Gu

ftware

ide, Depart

Project

ment of

20 DOE G 414.1-4

6-17-05

isks throughout all phases of the project’s life-cycle should include special

mphasis on tracking the risks associated with costs, resources, schedules, and technical aspects

clude—

• using unproven computer and software technologies such as programming languages not

i ;

The risks associated with the safety software applications need to be understood and

documented. The above bulleted list identifies a few potential risks associated with safety

software applications. Each risk should be evaluated against its risk thresholds. Different

Risk assessment and risk control are two fundamental activities required for project success. Ris

assessment addresses identification of the potential risks, analysis of those risks, and prioritizin

the risks to ensure that the necessary resources will be available to mitigate the risks. Risk

control addresses risk tracking and resolution of the risks. Identification, tracking, and

management of the r

of the project. Several risk identification techniques are described and detailed in standards and

literature.

17,18

Risk resolution includes risk avoidance, mitigation, or transference. Even the small risks during

one phase of the safety software application’s life have the potential to increase in some other

phase of the application’s life with very adverse consequences. In addition, mitigation actions fo

some risks could create new (secondary) risks.

Examples of potential software risks for the safety software application might in

• incomplete or volatile software requirements;

• specification of incorrect or overly simplified algorithms or algorithms that will be very

difficult to address within safety software;

• hardware constraints that limit the design;

• potential performance issues with the design;

• a design that is based upon unrealistic or optimistic assumptions;

• design changes during coding;

• incomplete and undefined interfaces;

ntended for the target application

• use of a programming language with only minimal experience using the language;

• new versions of the operating system;

• unproven testing tools and test methods;

• insufficient time for development, coding, and/or testing;

• undefined or inadequate test acceptance criteria; and

• potential quality concerns with subcontractors or suppliers.

Christensen, Mark J., and Richard

H. Thayer, The Project Manager’s Guide to Software Engineering’s Best

, pp. 417–447. Practices, Institute of Electrical and Electronics Engineers Computer Society Press, 2001

Society of Automotive Engineers (SAE) JA1003, Software Reliability Program Implementation Guide, SAE

2004, Appendix C4.6.

DOE G 414.1-4 21

6-17-05

isks.

ritized, resolved to an acceptable level of risk, and tracked through the life of the

fety software. For Level B or Level C software applications, the granularity for the risks to be

determine a graded approach for resolving the

sks and the process for tracking the risks.

This work activity does not apply to commercial design and analysis safety software because this

alysis services provided to DOE from a

rform

procure n or analysis requirements.

ng risk management is provided by IEEE

andar nce regarding the risk management of

isting organizational

k ma 1.00-1999, Software Risk Management: A Practical

risk transference, and risk avoidance that may be of

erest

2.3

ines

ange control process.

The following four areas

anagement:

control, (3) configuration status accounting,

E NQA-1-2000 software

audits and reviews.

techniques may be used to evaluate the risks. Examples of these techniques include decision

trees, scenario planning, game theory, probabilistic analysis, and linear programming. Various

treatment alternatives to addressing risk should be considered to avoid, reduce, or transfer r

Flexibility may need to be applied regarding risk management based upon the risk categorization

of the safety software application. For a Level A safety software, all apparent risks known at the

time, whether large or small, should be identified, analyzed for impact and probability of

occurrence, prio

identified, analyzed, prioritized, resolved to an acceptable level of risk, and tracked should be

determined by the safety system staff and can be graded. The safety system staff should focus o

the adverse events that would dominate the risk and assess these in a qualitative manner. The

safety system staff also has the responsibility to

software is used in conjunction with the design and an

commercial contractor. The risk management work activity associated with that software is

pe ed by the service supplier. DOE controls the SQA activities of that software through

ment agreements and specifications of desig

Further guidance beyond that in NQA-1-2000 regardi

St d 16085-2004.

This standard provides guida

acquired, developed, operational, or maintained systems to support the ex

ris nagement processes. SQAS21.0

Guide, also discusses a risk taxonomy,

int to the safety software analyst.

5. Software Configuration Management

SCM activities identify all functions and tasks required to manage the configuration of the

software system, including software engineering items, establishing the configuration basel

to be controlled, and software configuration ch

of SCM should each be addressed when performing configuration m

(1) configuration identification, (2) configuration

and (4) configuration audits and reviews. This Guide extends ASM

configuration management tasks by including configuration

s (IEEE)

EEE,

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 203, p. 105.

EE, 2003, Section 5.3.5.

International Organization for Standardization (ISO)/Institute of Electrical and Electronics Engineer

Std 16085, IEEE Standard for Software Engineering: Software Life Cycle Processes—Risk Management, I

2004.

IEEE Std 828-1998, IEEE Standard for Software Configuration Management Plans, IEEE, 1998, Section 4.3.

ASME NQA-1-2000, op. cit., Part I, Section 802, p. 16.

IEEE Std 7-4.3.2-2003, IEEE Standard Criteria for Digital Computers in Safety Systems of Nuclear Power

Generating Stations, IE

22 DOE G 414.1-4

6-17-05

of each

nd configuration reviews and audits.

are

nted

udits or reviews should be conducted to verify that the software product is consistent with the

Level A or Lever B, all four areas of SCM noted

a ap re graded as Level A or Level B and all Level C

ion

re are a

variety of approaches for software procurement and supplier management based upon—

•

service being procured and

The methods used to control, uniquely identify, describe, and document the configuration

version or update of software and its related documentation should be documented. This

documentation may be included in a SCM plan or its equivalent. Such documentation should

include criteria for configuration identification, change control, configuration status accounting,

During operations, authorized users lists can be implemented to ensure that the software use is

limited to those persons trained and authorized to use the software. Authorized users lists

access control specifications that are addressed in Section 5.2.5, Software Requirements

Identification and Management.

A baseline labeling system should be implemented that uniquely identifies each configuration

item, identifies changes to configuration items by revision, and provides the ability to uniquely

identify each configuration. This baseline labeling system is used throughout the life of the

software development and operation.

Proposed changes to the software should be documented, evaluated, and approved for release.

Only approved changes should be made to the software that has been baselined. Software

verification activities should be performed for the change to ensure the change was impleme

correctly. This verification should also include any changes to the software documentation.

configuration item descriptions in the requirements and that the software, including all

documentation, being delivered is complete. Physical configuration audits and functional

configuration audits are examples of audits or reviews that should be performed.

SCM work

activities should be applied beginning at the point of DOE’s or its contractor’s control of the

software.

For custom developed safety software graded at

bove ply. For all other types of safety softwa

safety software, this work activity may be graded by the optional performance of configurat

audits and reviews.

5.2.4

Procurement and Supplier Management

Most software projects will have procurement activities that require interactions with suppliers

regardless of whether the software is Level A, B, or C. Procurement activities may be as basic

the purchase of compilers or other development tools for custom developed software or as

complicated as procuring a complete safety system software control system. Thus, the

the level of control DOE or its contractors have on the quality of the software or software

• the complexity of the software.

Institute of Electrical and Electronics Engineers (IEEE) 1042-1987, IEEE Guide to Software Configuration

Management, IEEE, 1987, Section 3.3.4.

DOE G 414.1-4 23

6-17-05

rocurement documentation should include the technical

and quality

requirements for the

g and validating the software, including any

documentation to be delivered;

e supplier,

on the complexity of the software and its importance to safety.

em requirements should be translated into requirements specific for the

s re be documented in system level requirements

docume ocurement contracts, and/or other acquired

safety software. Some of the specifications that should be included are—

• specifications for the software features, including requirements for safety, security,

functions, and performance;

• process steps used in developin

• requirements for supplier notification of defects, new releases, or other issues

that

impact the operation; and

• mechanisms for the users of the software to report defects and request assistance i

operating the software.

These requirements should be assessed for completeness and to ensure the quality of the softwar

being purchased. There are four major approaches for this assessment:

• performing an assessment of th

• requiring the supplier to provide a self-declaration that the safety software meets the

intended quality,

• accepting the safety software based upon key characteristics (e.g., large user base), and

verifying the supplier has obtained a certification or accreditation of the software product •

quality or software quality program from a third party (e.g., the International

Organization for Standardization, Underwriters Laboratories, and Software Engineering

Institute).

It is important to note that while Levels A, B, and C software applications are required to fully

meet this work activity, the implementation detail and assessment method of the supplier can

vary based

5.2.5

Software Requirements Identification and Management

Safety system requirements provide the foundation for the requirements to be implemented in th

software. These syst

oftwa . The identified software requirements may

nts, software requirements specifications, pr

software agreements. These requirements should identify functional; performance; security,

including user access control; interface and safety requirements; and installation considerations

and design constraints where appropriate. The requirements should be complete, correct,

consistent, clear, verifiable, and feasible.

NQA-1-2000, op. cit., Part I, Req

NQA-1-2000, op. cit., Part I, Requir

ASME uirement 4, Section 202, p. 18.

ASME ement 4, Section 100, p.18.

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 301, p. 105.

ware

Institute of Electrical and Electronics Engineers (IEEE) Std 830-1998, IEEE Recommended Practice for Soft

Requirements Specifications, IEEE, 1998, Section 4.3.

24 DOE G 414.1-4

6-17-05

an important aspect to ensuring only authorized users

can operate the system or use the software for design or analysis tasks. Controlling access is a

fications as part of the

Once th been defined and documented, they should be managed to

nimi ies to ensure

the corr to operations. Software requirements should be traceable

This work activity has no grading associated with its performance. Software requirements

and sho rement. However, the detail and format of the safety software

requirements may vary with the software type. Custom developed software most likely will

ore formal document

may be applicable.

During software design and implementation the software is developed, documented, reviewed,

require escribe

how th

function internally. Data structure requirements and layouts may be necessary to fully understand

the internal operations of the software.

ionships between data

lements, interfaces with external components, and basic database table structures may be all that

are need m developed software,

User access control during operations is

software safety and/or security requirement that can be associated with training or qualification

to operate the system. ASME NQA-1-2000 addresses access control speci

operations phase.

e software requirements have

mi ze conflicting requirements and maintain accuracy for later validation activit

ectness of the software placed in

throughout the software life-cycle.

identification management and traceability applies to Level A, B, and C software applicatio

uld fully meet this requi

contain a larger number of software requirements than configurable, acquired, utility calculation,

or commercial design and analysis tool software, and thus, a separate m

5.2.6 Software Design and Implementation

and controlled. The software design elements should identify the operating system, function,

interfaces, performance requirements, installation considerations, design inputs, and design

constraints. The software design should be complete and sufficient to meet the software

ments.

The design activities and documentation should be adequate to fully d

e software will interface with other system components

and how the software will

Custom developed software will require more formality in the documentation and review of th

design than configurable or utility calculations. Simple process flows, relat

ed for configurable or utility calculations, whereas for custo

complete functional and logical designs of the software components, the input and output data,

and pseudo code may be required to fully understand the safety software design. The software

design description may be combined with the documentation of the software requirements or

software source code.

During implementation, static analysis, clean room inspections, and reviews are common

techniques to ensure the implementation remains consistent with the design and does not add

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 405, p. 106.

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 401, p. 106.

ASME NQA-1-2000, op. cit., Part I Introduction, and Section 801.2, p. 1

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 402, p. 106.

ASME NQA-1-2000, op. cit., Part I Introduction and Section 801.2, p. 16.

DOE G 414.1-4 25

6-17-05

omplexity or functions which could decrease the safe operation of the software. Many tools

he software developer should perform unit testing prior to system level V&V techniques,

a graded or tailored approach to ensure the known risks are

ng;

ign

performed. Additionally, formal developer testing that includes functional,

structural, tim ctors testing should be planned, performed and

ple

at includes safety functions, security, and performance

ign strategies to eliminate or mitigate those hazards. Hence, it is

commended that the software safety process address the mitigation strategy for the components

that have potential safety consequences if a fault occurs, whereas the software design and

im re of the safety software application.

exist to evaluate the complexity and other attributes of the source code design structure.

Walkthroughs and more formal inspections, such as Fagan inspections, can be used to identify

defects in source code, as well as design descriptions and other software development process

outputs.

including acceptance testing. Developer testing can be very structured and formal, using

automated tools or less formal methods. In addition to unit testing, functional, structural, timing

(performance testing), stress, security, and human-factors testing are useful testing methods.

These methods can be applied using

mitigated appropriately. Other techniques

34,35

such as error seeding; equivalence class testi

branch and path testing; statistical-based, boundary value testing; and code coverage analysis

may all be beneficial testing techniques to ensure robust and reliable software.

The software design and implementation work activity for Levels A, B, and C custom developed

software applications should fully meet this requirement. For this software type, the design,

including interfaces and data structures, should be completely documented; reviews of the des

and code should be

ing, stress, security, and human-fa

the results documented. It is recommended that the complexity of the custom developed safety

software be evaluated and analysis performed to reduce the complexity of the source code

modules.

Configurable and utility calculation for Levels A, B, and C software applications may be grad

for this work activity. This grading should include fully performing the design work activities

with custom developed software. However, less formal design and code reviews, such as sim

desk checks by another individual other than the developer, may be performed. Developer testing

should be performed and documented th

testing. This work activity does not apply to acquired or commercial design and analysis safety

software types since the design and implementation activities associated with commercial design

and analysis software are performed by the service supplier. DOE controls the SQA activities of

that software through procurement agreements and specifications.

5.2.7

Software Safety

The development of software applications requires identification of hazards (i.e., abnormal

conditions and events) that have the potential for defeating a safety function and the

mplementation of desi

plementation process addresses the architectu

Pressman, Roger S., Software Engineering: A Practitioner’s Approach, McGraw Hill, 1992, pp. 595–629.

fety and Reliability, Lawrence Livermore

Sparkman, Debra, Techniques, Processes, and Measures for Software Sa

National Laboratory, UCRL-ID 108725, 1992.

26 DOE G 414.1-4

6-17-05

level.

se a system failure.

uring the initial concept and requirement analysis phases for the software, potential failures

ut the requirements and design structure. These techniques include failure modes and

ffects analysis, fault-tree modeling, event-tree modeling, cause-consequence diagrams, hazard

horoughly. Separation of the

safety features also allows for more rigorous software development and verification practices to

be applied to the safety components while providing the appropriate and cost effective level of

uld

upt

Software is only one component of the overall safety system. It may be embedded in an I&

system, it may be a custom control system for hardware components, or it may be standalone

software used in safety management or support decisions. In any of these or other applications

software important to safety, analysis of the software application occurs first at the system

The analysis should then be performed at the software component level to ensure adequate

safeguards are provided to eliminate or mitigate the potential occurrence of a software defect th

could cau

Methods to mitigate the consequences of software failures should be an integral part of the

software design.

Specific software analysis and design methods for ensuring that safety

functions are well thought out and addressed properly should be performed throughout the

software development and operations life-cycles. These methods include dynamic and static

analyses. The techniques and methods described in this section are only a selection of those

available. Several resources are available to assist in the selection and use of these methods.

A few are listed in the reference section of this Guide.

need to be identified and evaluated for their consequences of failure and probability of

occurrence. Some potential problems are (1) complex or faulty algorithm, (2) lack of proper

handling of incorrect data or error conditions, (3) buffer overflow, and (4) incorrect sequence of

operations due to either logic or timing faults.

There are several hazard analysis techniques that may be used for this purpose. Many of these

techniques are performed as preliminary analyses and later updated as more information is

known abo

and operability analysis, and interface analysis. Techniques such as these should be applied and

appropriately documented to understand and assess the impact of software failures on the system.

The design of the software is critical to ensuring safe operation of the system. The software

design should consider principles of simplicity, decoupling, and isolation to eliminate the

hazards.

Complexity of the software design, including the logic and number of data inputs, has

proven to increase the defect density in software components. The safety features should be

separate from nonsafety modules, minimizing the impact of failure of one module on another.

The interfaces between the modules need to be defined and tested t

SQA applied to the nonsafety components. Software engineering safety design practices sho

include process flow analysis, data flow analysis, path analysis, interface analysis, and interr

analysis during the design phase.

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 402, p. 106.

Leveson, op. cit., pp. 400–412.

IEEE Std. 7-4.3.2-2003, op. cit., Section 5.6, p. 1

DOE G 414.1-4 27

6-17-05

uced

e the

apabilities of the overall system that may not be immediately detectable by the system. In these

ely

ory functionality and integrity tests, such as checksums and watch

dog timers for software processes, including operating system processes.

Additionally,

ing

for

d. For Level A custom developed safety

ftware, the design concepts that include simplicity of modules that perform safety functions

ault

Level B or Level C software applications may be

raded. This grading may include fully performing the safety analysis activities for the software

ethods described above could add undue burden to the development of these applications and

development or acquisition processes to ensure the software meets the intended requirements.

When hazards related to software functions cannot be eliminated, the hazard should be red

and/or monitored. Additionally, software can experience partial failures that can degrad

instances, other design techniques, such as building fault detection and self-diagnostics into th

software, should be implemented. Using external monitors (safety bag) for the software safety

functions, n-version programming, and Petri nets are examples of techniques

39,40

that can ensure

the software design adequately addresses safety issues and minimizes failure modes by adding

fault tolerant concepts. Self-diagnostics detect and report software faults and failures in a tim

manner and allow actions to be taken to avoid an impact on the system operating safety. Some of

these techniques include mem

software control functions can be performed incrementally rather than in a single step, reduc

the potential that a single failure of a software component would cause an unsafe state.

The software safety work activity for Level A custom developed, configurable, and acquired

safety software should fully meet this requirement. For this software type the safety analysis

the software components should be performed. This analysis may be part of the overall safety

system analysis if detailed software failures are include

and isolation of those modules should be part of the design considerations. Where the design o

the software modules still presents an unacceptable risk to failure of the safety system, f

tolerant and self-diagnostics designs should be implemented.

Custom developed, configurable, and acquired

components to ensure the safety aspects are being addressed. The design concepts of simplicity

and isolation and fault tolerance and self-diagnostics may not apply to Level B or Level C

software applications and, thus, can optionally be applied.

This work activity does not apply to utility calculation or commercial design and analysis safety

software types. Utility calculations are typically simple calculations where techniques and

not increase the assurance that any software failure would not impact safety. However, if the

safety analysis determines that complexity of the utility calculation warrants the use of thes

techniques, they should be applied. For commercial design and analysis software, the software

safety activities are performed by the service supplier. DOE controls the SQA activities of th

software through procurement agreements and specifications.

5.2.8

Verification and Validation

V&V is the largest area within the SQA work activities. Verification is performed throughout th

life-cycle of the safety software. Validation activities are performed at the end of the software

Sparkman, op. cit.

SAE JA1003, op. cit., Appendix C

IEEE Std 7-4.3.2-2003, op. cit., Section 5.5.3, p. 13.

28 DOE G 414.1-4

6-17-05

activities include reviews, inspections, assessments,

bservations, and testing. This Guide expands on ASME NQA-1-2000 acceptance testing

n may

upplier assessments are important aspects of V&V. Assessments are covered in Section 5.2.4,

and all test activity deliverables

laced under configuration management.

g strategies that may be appropriate

for acceptance testing include equivalence class testing, branch and path testing, statistical-based

V&V activities should be performed by competent staff other than those who developed the item

being verified or validated.

V&V

activities to include more extensive V&V activities of reviews, inspections, assessments, and

observations as described in other consensus standards.

Reviews and inspections of software deliverables requirement specifications, procurement

documents,

software design, code modules, test results, training materials, user documentatio

and processes that guide the software development activities should be performed. The software

deliverables may be combined with other software or system documents. Traceability of the

software requirements to the software design should be performed.

As mentioned in the

development practice section, inspections can be formally implemented Fagan inspections,

walkthroughs, or desk checks. Verification of the software design, using one of the above

methods, should be completed prior to approval of the software for use.

This verificatio

be performed as part of the software development and implementation activity.

Procurement and Supplier Management, and Section 6, Assessment and Oversight.

Observations and testing can be performed during the development, factory or site acceptance

testing, installation, and operation (i.e., in-use testing)

of the software. Observations and testing

during development is discussed in Section 5.2.6, Software Design and Implementation.

Software testing activities should be planned and documented. Test cases and procedures,

including expected results, should be created. All test activity deliverables should be under

configuration management. Test results should be documented

Acceptance testing should include functional testing, performance testing, security testing, stress

testing, and load testing. Users’ guides, use cases, and operational profiles are instrumental in

identifying and detailing the positive test cases and procedures. Failure mode analyses can b

used for defining negative test cases and procedures. Testin

and boundary value testing.

Additionally, the system should continually be monitored to estimate its continuing reliability

and safety. Periodic testing of the operational system should be performed to detect any

degradation.

If testing is not possible, monitoring using quantitative measurements should b

performed.

ASME NQA-1-2000, op. cit., Part I, Requirement 3, Section 801.1, p. 16.

ASME NQA-1-2000, op. cit., Part I, Requirement 4, Section 300, p. 18.

ASME NQA-1-2000, op. cit, Part II, Subpart 2.7, Section 402.1, p. 106.

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 402.1, p. 106.

0, op. cit., Part I, Requirement 11, Section 400, p. 29.

Requirement 11, Section 200, p. 29.

ction 400, p. 30.

ASME NQA-1-200

ASME NQA-1-2000, op. cit., Part I,

ASME NQA-1-2000, op. cit., Part I, Requirement 11, Se

DOE G 414.1-4 29

6-17-05

When a new version of a software product is obtained, predetermined and ad-hoc test cases and

users with nearly the same capabilities

and

that are reused

deliverables

ing

tance test cases and procedures, including expected

or Level A software, continual monitoring of safety software operations based upon historical

failure data and results of periodic reassessment of hazards should be performed. For Level A, B,

n developed, reviews and

e performed.

procedures should be performed to validate that the system meets the requirements and does not

perform any unintended functions.

If the system is operational, only positive testing may be

possible. In those instances, it is important to perform analysis of failure modes for the software

to understand the consequences if the software or system should get into an abnormal operational

state.

Modern utility calculation applications, such as spreadsheet programs, have grown dramaticall

in power, with a corresponding growth in risk. The addition of macro programming languages

nd the ability to incorporate “add-in” programs provide a

as code developed with traditional programming tools. Utility calculation applications are

installed on virtually every desktop, and user files containing algorithms and data can be easily

modified by users. Section 101.1 of ASME NQA-1-2000, Subpart 4.1, provides useful guidance

on V&V of utility calculations. Calculations performed using applications such as commercial

spreadsheet programs may be treated in either of two ways. In the case of relatively

straightforward calculations, the calculation result may be checked and verified in the same

manner as a hand calculation. For more complex or extensive calculations, where checking

verification of calculation results are impractical or undesirable, the user files containing the

calculation formulas, algorithms, or macros should be subject to the entire software life-cycle

rocess. The latter approach may also be expedient for calculation applications p

frequently.

ustom developed software will most likely have a larger number and more detailedC

than would utility calculations. For Level A safety software all deliverables should be reviewed

using V&V methods. Additionally for Level A, traceability of the requirements to the design and

from requirements to test cases should be performed. For Level B safety software, deliverables

that include requirements, test plans and procedures, and test results should be reviewed us

V&V methods.

For all Level A safety software except utility calculations, acceptance testing work activities

hould be planned and documented; acceps

results should be created; test results should be documented; and all test activity deliverables

should be under configuration management. Level A utility calculations and Level B and C

custom developed, configurable, acquired, and utility calculations can use a graded approach b

applying less formality in the documentation. Simple check lists for acceptance test cases and

procedures may be used in place of more detailed test cases and procedures. Test results should

be documented and all test activity deliverables placed under configuration management.

or C software, when new releases of the safety software have bee

bacceptance testing of changed documents and software should

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 404, p. 106.

ASME NQA-1-2000, op. cit., Section 101.1, Subpart 4.1.

30 DOE G 414.1-4

6-17-05

ed by

and specifications.

enting, evaluating and correcting software problems; (2) an evaluation process for

etermining whether a reported problem is indeed a defect or an error; and (3) the roles and

and

ems to the

s to

need not be separate from the other problem reporting and corrective action

rocesses if the existing process adequately addresses the items in this work activity.

that reduces the formality of documenting problem reports and approving

orrective actions taken may be applied for Level A and B utility calculation safety software and

SQA

This work activity does not apply to commercial design and analysis safety software types sinc

the V&V activities associated with commercial design and analysis software are perform

the service supplier. DOE controls the SQA activities of that software through procurement

agreements

5.2.9

Problem Reporting and Corrective Action

Coupled with the configuration management of the software system, the problem reporting and

corrective action process should address the appropriate requirements of the QAP corrective

action system. The reporting and corrective action system will cover (1) methods for

docum

responsibilities for disposition of the problem reports, including notification to the originator of

the results of the evaluation.

If the noted problem is indeed an error, the problem reporting and

corrective action system should correlate the error with the appropriate software engineering

elements; identify the potential impacts and risks to past, present, and future developmental

operational activities; and support the development of mitigation strategies. After an error has

been noted, all users should be apprised to ascertain any impacts upon safety basis decisions.

Procurement documents should identify the requirements for suppliers to report probl

supplier, any required supplier response, and the method for the purchasers to report problem

the supplier.

Maintaining a robust problem reporting and corrective action process is obviously vital to

maintaining a reliable and vital safety software system. This problem reporting and corrective

action system

This work activity should be fully implemented for all Level A and B software types (custom

developed, acquired, configurable, and commercial design and analysis) and for Level C custom

developed. This formal implementation should include documentation and tracking to closure of

any problems reported for the software and authorization to perform the corrective action. A

graded approach

all Level C software applications except custom developed. This less formal implementation

may include interoffice communications describing the problem identified and the corrective

actions planned.

5.2.10

Training Personnel in the Design, Development, Use, and Evaluation of Safety

Software

Training personnel in designing, developing, testing, evaluating, or using the safety software

application is critical for minimizing the consequences of software failure. Although other

204, p. 229.

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 204, p. 105.

ASME NQA-1-2000, op. cit., Part II, Subpart 2.7, Section 301, p. 105.

ASME NQA-1-2000, op. cit., Part IV, Subpart 4.1, Section

DOE G 414.1-4 31

6-17-05

ork activities may indicate that the software satisfies its operational objective, improper or

ation users, and

perations staff. The analyst and developers may need training in fault tolerant methodologies,

ed, that proper options and menus are selected, and that the

results of the software can be interpreted correctly. A trained and knowledgeable staff is

e ia to ensure the proper levels of quality and

te in

For Level B and C software applications, this work activity can be graded to

onal,

and

management and control issues.

invalid use of the software may negate the safety mitigation strategies included within the

software.

Training may be necessary for the analyst, development and test teams, applic

safety design methodologies, user interface design issues, testing methodologies, or

configuration management to ensure delivery of a robust software application. Meanwhile, the

software application users and operations staff may need training specific to the software to

ensure that proper data are enter

ssent l to assess and evaluate the SQA requirements

safety exists in the software.

Training should be commensurate with the scope, complexity, and importance of the tasks an

the education, experience, and proficiency of the individual. Indoctrination as described in

ASME NQA-1-2000

meets this work activity requirement. Personnel should also participa

continuing education and training as necessary to improve their performance and proficiency and

ensure that they stay up-to-date on changing technology and new requirements.

Completion of training, education, and/or qualification requirements for all staff involved in the

development, testing, use, and evaluation of custom developed or configurable software graded

as Level A, B, or C should be documented and reviewed periodically. This may include a

osition description, qualification criteria, or a list of training courses along with verification of p

successfully meeting the knowledge requirements. Completion of training, education, and/or

qualification requirements for all staff involved in the procurement, testing, use, and evaluation

of acquired or utility calculation software graded as Level A should be documented and reviewed

eriodically.p

include periodic evaluation by the appropriate supervising authority of the training, educati

or qualification requirements for performing assigned tasks associated with using and evaluatin

acquired or utility calculation software. This work activity does not apply to commercial desig

and analysis safety software since the training activities associated with commercial design

nalysis software are performed by the service supplier. DOE controls the SQA activities of that a

software through procurement agreements and specifications.

6. ASSESSMENT AND OVERSIGHT

6.1 GENERAL

DOE assessment requirements in 10 CFR 830 Subpart A and DOE O 414.1C should be applie

to safety software

ASME NQA-1-2000, op. cit., Part I, Requirement 2, Section 200, p. 10.

ea Qualification Standard, dated 12-03.

DOE-STD-1172-2003, Safety Software Quality Assurance Functional Ar

32 DOE G 414.1-4

6-17-05

6 D

DOE

ine ES&H Oversight Policy, dated 5-31-01, contains guidance on independent and

odel

for the work will ensure that the SQA implementation process

ctions related

safety software issues.

.2 OE AND CONTRACTOR ASSESSMENT

DOE should assess the effectiveness of its actions in resolving issues related to safety softwar

management and controls. DOE also evaluates the adequacy and implementation effectiveness

of DOE and contractor safety software management and controls. DOE G 414.1-1, Management

Assessment and Independent Assessment Guide for Use with 10 CFR, Part 830, Subpart A, an

DOE O 414.1A, Quality Assurance; DOE P 450.4, Safety Management System Policy; and

450.5, LP

management assessment.

Contractors are expected to assess the adequacy and effectiveness of their safety software

controls in accordance with DOE O 414.1C and this Guide.

A model criteria review and approach document (CRAD) is provided in Appendix F. This m

contains software qualification assessment criteria for assessing the safety software used for

safety analysis and design of safety SSCs and I&C systems in the defense nuclear facilities.

The organization responsible

addresses the processes presented in this Guide.

6.3 DOE INDEPENDENT OVERSIGHT

The DOE Office of Independent Oversight and Performance Assurance, and the Office of

Inspector General are responsible for conducting independent oversight of DOE a

The DOE/NNSA SQA responsible person will verify that the SQA implementation process

meets the intent of this Guide throughout the entire software life-cycle as described in the Q

and procedures.

DOE G 414.1-4 APPENDIX A

6-17-05 A-1

APP NS

A.1. ACRONYMS

al Standards Institute

ASME American Society of Mechanical Engineers

r Quality Control

Defense Nuclear Facilities Safety Board

DoD U.S. Department of Defense

gy Order

IEC International Electrotechnical Commission

lectrical and Electronics Engineers

National Aeronautics and Space Administration

NNSA National Nuclear Security Administration

quality assurance program

QARD quality assurance requirements document

RSICC Radiation Safety Information Computational Center

SAE Society of Automotive Engineers

SC safety class

SCM software configuration management

SDD software design description

SDP software development plan

ENDIX A. ACRONYMS AND DEFINTIO

ANS American Nuclear Society

ANSI American Nation

ASQC American Society fo

CFR Code of Federal Regulations

CMMI Capability Maturity Model Integration

COTS commercial off-the-shelf

CRAD criteria review and approach document

DCS distributed control system

DNFSB

DOE G U.S. Department of Energy Guide

DOE O U.S. Department of Ener

DOE U.S. Department of Energy

DSA documented safety analysis

HMI human-machine interface

I&C Instrumentation and control

IAEA International Atomic Energy Agency

IEEE Institute of E

ISO International Organization for Standardization

M&O management and operating

NASA

PLC programmable logic controller

QA quality assurance

QAP

APPENDIX A DOE G 414.1-4

A-2 6-17-05

SEI Software Engineering Institute

SG safety guide

SMS safety management system

SPMP software project management plan

SQA software quality assurance

SQAP software quality assurance plan

SRS software requirement specification

SS safety significant

SSC structure, system, and component

SW-CMM Software Capability Maturity Model

TR technical report

TSR technical safety requirement

USQ unreviewed safety question

USQD unreviewed safety question determination

V&V verification and validation

VV&A verification, validation, and accreditation

WIPP Waste Isolation Pilot Project

DOE G 414.1-4 APPENDIX A

6-17-05 A-3

The following definitions are included with this Guide for convenience and clarification. DOE

O 414.1 nitions sh ll take pre in this appendix.

Acceptance Testing. T process o stem component by

manual or automated means to ensu and to identify

differen een exp cted and a

NQA-1-2000.

Admini Contr . The pro agement, procedures,

record k assessm safe operation of a facility.

Source: 10 CFR 830.

Assessm view, valuation, dit, to determine

and doc ether ems, proce requirements and

perform ively. So ce: DOE O

Configuration Management. The the configuration items in

a system software d hardwa d change of these items

throughout the system’s ife cycle, he status of configuration items

and change requests. So rce: ASM

Consequence. An outcome of an ev n. Source: IEEE

Std 1540-2001.

Firmwa combin tion of a h a that reside

as read-only software on that devic only to the

hardware device or only to the com e deprecated.

(2) The on surro nding this t that it be avoided altogether.

Source: d 610.1 -1990.

Functional Configuration Audit. An audit conducted to verify that the development of a

configuration item has been comple as achieved the performance

and func l characte stics speci al or allocated configuration identification,

and that rational d support d satisfactory. Source:

IEEE Std-610.12-1990.

Graded Approach. Th process of d

actions used to comply ith require

• the relative importance to safety, safeguards, and security;

• t nitude o any hazard

• t -cycle sta e of a faci

• the programmatic mission o

• the particular characteristics of a facility or item;

A.2. DEFINITIONS

C defi a cedence over those included

he f exercising or evaluating a system or sy

re that it satisfies the specified requirements

ces betw e ctual results in the operating environment. Source: ASME

strative ols visions relating to organization and man

eeping, ent, and reporting necessary to ensure

ent. A re

e inspection, test, check, surveillance, or au

t specified ument

effect

sses, systems, or services mee

414.1C.

process of identifying and defining

(i.e., an re), controlling the release an

and recording and reporting t

E NQA-1-2000.

ent, hazard, threat, or situatio

re. The a ardware device and computer instructions and dat

e. Notes: (1) This term is sometimes used to refer

puter instructions or data, but these meanings ar

confusi u term has led some to sugges

IEEE St 2

ted satisfactorily, that the item h

tiona ri fied in the function

its ope an documents are complete an

e ensuring that the level of analyses, documentation, an

w ments are commensurate with—

he mag f involved;

he life g lity or item;

f a facility;

APPENDIX A DOE G 414.1-4

A-4 6-17-05

• t ative imp tance to ra azards; and

• any other relevant factors.

Source: 10 CFR 830.

Hazard Controls. Me ures to eli workers, the public, or

the envi nt, includ g— 10 CF

(1) physical, design, structural,

(2) safety structures, systems an

(3) safety managem nt program

(4) Technical Safety Requireme

(5) o sary to p rds.

Source: 10 CFR 830.

Item. An all-inclusive term used in bly, component, equipment,

material le, part, ructure, pr , subsystem, system, unit, or

support Source 10 CFR 8

Nuclear Facility. A rea tor or a no re an activity is conducted for or

on behalf of DOE and includes any to the extent

necessary to ensure proper impleme tablished in CFR, part 10,

ction 830. Source: 10 CFR 830.

Physical Configuration Audit. An audit conducted to verify that a configuration item, as built,

conforms to the technical documentation that defines it. IEEE Std-610.12-1990.

Process. A series of actions that achieves an end result. Source: 10 CFR 830.

Quality. The condition achieved when an item, service, or process meets or exceeds the user’s

requirements and expectations. Source: 10 CFR 830.

Quality Assurance. All those actions that provide confidence that quality is achieved. Source:

10 CFR 830.

Quality Assurance Program. The overall program or management system established to assign

responsibilities and authorities, define policies and requirements, and provide for the

performance and assessment of work. Source: 10 CFR 830.

Risk. The likelihood of an event, hazard, threat, or situation occurring and its undesirable

consequences; a potential problem. Source: IEEE Std 1540-2001.

Safety. An all-inclusive term used synonymously with environment, safety, and health to

encompass protection of the public, the workers, and the environment. Source: DOE O 414.1C.

Safety-class structures, systems, and components (SC SSCs). Structures, systems, or

components, including portions of process systems, whose preventive and mitigative function is

he rel or diological and nonradiological h

as minate, limit, or mitigate hazards to

ronme in R 830

and engineering features;

d components

e s;

nts; and

ther controls neces rovide adequate protection from haza

place of appurtenance, assem

, modu st oduct, software, subassembly

systems. : 30.

c nreactor nuclear facility whe

related area, structure, facility, or activity

ntation of the requirements es

DOE G 414.1-4 APPENDIX A

6-17-05 A-5

necessary to limit radioactive hazard a ined from the

safety analyses. Source: 10 CFR 830.

or mitigative

nction is a major contributor to defense in depth and/or worker safety as determined from

diological or chemical exposure to workers. Source: DOE G 420.1-1

the

ards analysis of nuclear facilities or an SSC that performs a safety

nction. Source: DOE O 414.1C.

r radiological safety management programs or

echnical Safety Requirements or other software that performs a control function necessary to

ment as

afety Management Program. A program designed to ensure a facility is operated in a manner

ic such as:

; maintenance of safety systems; personnel training; conduct of operations;

advertent criticality protection; emergency preparedness; fire protection; waste management;

n as part

component and is cited in either DOE approved documented safety

nalysis or an approved hazard analysis per DOE P 450.4, Safety Management System Policy,

tion, environmental

medi software development/

ous m terial exposure to the public, as determ

Safety-significant structures, systems, and components (SS SSCs). Structures, systems, and

components which are not designated as safety-class SSCs, but whose preventive

safety analyses [10 CFR 830]. As a general rule of thumb, safety-significant SSC designations

based on worker safety are limited to those systems, structures, or components whose failure is

estimated to result in a prompt worker fatality or serious injuries (e.g., loss of eye, loss of limb)

or significant ra

Safety and Hazard Analysis Software and Design Software. Software that is used to classify,

design, or analyze nuclear facilities. This software is not part of an SSC but helps to ensure

proper accident or haz

Safety Management and Administrative Controls Software. Software that performs a hazar

control function in support of nuclear facility o

provide adequate protection from nuclear facility or radiological hazards. This software supports

eliminating, limiting, or mitigating nuclear hazards to workers, the public, or the environ

addressed in 10 CFR 830, 10 CFR 835, and the DEAR ISMS clause. Source: DOE O 414.1C.

that adequately protects workers, the public, and the environment by covering a top

quality assurance

or radiological protection of workers, the public, and the environment. Source: 10 CFR 830.

Safety Software. Includes safety system software, safety and hazard analysis software and

design software and safety management and administrative controls software. Source: DOE

O 414.1C.

Safety Structures, Systems, and Components. Both safety class structures, systems, and

components and safety significant structures, systems, and components. Source: 10 CFR 830.

Safety System Software. Software for a nuclear facility

that performs a safety functio

of a structure, system or

dated 10-15-96, and the DEAR clause. Source: DOE O 414.1C.

Service. Work, such as design, construction, fabrication, decontamina

ple analysis, safetyre ation, waste management, laboratory sam

P CFR 830, quality assurance requirements apply toer 10 clear facilities including radiological facilities

e 10 C

all DOE nu

(se FR 830, DOE Std 1120, and the DEAR clause).

APPENDIX A DOE G 414.1-4

A-6 6-17-05

qualification,

uipm ssessment, repair, and installation or the like. Source:

: NQA-1-2000

lated actions that establish the

entified in the documents safety analysis for the

e and

ell as a bases

CFR 830.

the previous phase, and the final system or component

tal

ards and security; or data collection and analysis. Source:

validation/testing, inspection, nondestructive examination/testing, environmental

eq ent qualification, training, a

10 CFR 830.

Software. Computer programs, procedures, and associated documentation and data pertaining t

the operation of a computer system. Source

rols, and reTechnical Safety Requirements. The limits, cont

ns for the safe operation of a nuclear facility and include, specific parameters and requisite actio

as appropriate for the work and the hazards id

facility: safety limits, operating limits, surveillance requirements, administrativ

use and application provisions, and design features, as wmanagement controls,

ppendix. Source: 10 a

Verification and Validation. The process of determining whether the requirements for a syste

or component are complete and correct, the products of each development phase fulfill the

equirements or conditions imposed by r

complies with specified requirements. Source: IEEE Std-610.12-1990.

Work. A defined task or activity; such as research and development; operations; environmen

remediation; maintenance and repair; administration; safety software development, validation

esting, and use; inspection; safegut

DOE O 414.1C.

DOE G 414.1-4 APPENDIX B

6-17-05 B-1 (and B-2)

APPENDIX B. PROCEDURE FOR ADDING OR REVISING SOFTWARE TO

.............B-3

.......B-4

B.2.1 Adding Software ApplicatioNs to the Central Registry ......................................B-4

OR DELETING SOFTWARE FROM

THE DOE SAFETY SOFTWARE CENTRAL REGISTRY

CONTENTS

B.1 INTRODUCTION ...........................................................................................................B-3

B.1.1 Purpose....................................................................................................

B.1.2 Scope....................................................................................................................B-3

B.1.3 Functions..............................................................................................................B-

B.2 PROCESS .................................................................................................................

B.2.1.1 Evaluation Process .............................................................................B-7

B.2.1.2 Submittal to the Central Registry......................................................B-1

B.2.2 Revisions to Software Applications in the Central Registry .............................B-12

B.2.3 Removal of Software Applications from the Central Registry.........................B-

B.2.4 Issue Resolution and Action Communication....................................................B-13

B.3 REFERENCES ..............................................................................................................B-

DOE G 414.1-4 APPENDIX B

6-17-05 B-3

PROCEDURE FOR ADDING OR REVISING SOFTWARE TO OR DELETING

SOFTWARE FROM THE DOE SAFETY SOFTWARE CENTRAL REGISTRY

B.1.1 PURPOSE

manage

Registr

the Centr

DOE O

determine s

also pre

software from

More detailed

softwar

http://w

B.1 INTRODUCTION

Toolbox codes represent a small number of standard computer models or codes supporting DOE

safety analysis having widespread use and of appropriate qualification that are maintained,

d and distributed by DOE’s Safety Software Central Registry (referred to as the Central

y). The purpose of this appendix is to outline the procedure for adding new software to

al Registry that is consistent with software quality assurance (SQA) requirements of

414.1C, Quality Assurance,

Criteria are referenced for demonstrating compliance with

applicable SQA requirements, and are recommended for use in an evaluation process to

uitability of candidate software for inclusion in the Central Registry. Information is

sented in brief on the procedures to (1) revise or update toolbox software and (2) remove

the Central Registry due to retirement by the software developer.

information of the SQA requirements and criteria that are applicable to safety

e as a basis for consideration to the Central Registry is found at

ww.eh.doe.gov/sqa/central_registry.htm.

B.1.2 SCOPE

safety-related purpose that is proposed for inclusion in the Central Registry.

B.1.3 FUNCTIONS

Procedures to identify, document and submit additional software applications to the Central

Registry are based on the process followed to evaluate the six initial toolbox codes.

Following

this precedent, three principal entities described below perform the major tasks.

Software Sponsor—either the originator of the software (developer) or the primary user (site

organization) who is requesting the software to be placed in the Central Registry or a

combination of the two. In either case, this party is responsible for documenting SQA programs,

procedures and processes associated with development of the software, maintaining and

configuration controlling the software, developing new versions of the subject software,

addressing user questions, and resolving technical and programmatic issues. The software

sponsor is responsible for documenting the rationale for adding the subject software to the

Central Registry.

The scope of this procedure includes any software application used by DOE or its contractors for

DOE O 414.1C, Quality Assurance, dated 6-17-05.

U.S. Department of Energy, Implementation Plan for Defense Nuclear Facilities Safety Board Recommendation

2002-1: Quality Assurance for Safety Software at Department of Energy Nuclear Facilities, Report, dated 3-13-03.

APPENDIX B DOE G 414.1-4

B-4 6-17-05

SQA Evaluator—an independent reviewer of the computer software, who is not affiliated with

the software developing organization. It is required that the review organization or individuals

have a thorough understanding of the applicable SQA requirements, expert level knowledge and

application experience with the software in question, and an awareness of the overall context for

the use of the subject software as part of the DOE safety process. The SQA evaluator is

responsible for documenting the SQA evaluation of the candidate software, and based on this

evaluation, confirms that the software SQA satisfactorily meets requirements for inclusion to the

Central Registry.

DOE Office of Environment, Safety and Health, Office of Quality Assurance Programs—

reviews the candidate software SQA evaluation and decides whether the candidate software

should be included in the Central Registry.

Independence between the evaluator and the sponsor is critical for completion of a formal SQA

evaluation, and should be maintained throughout the Central Registry submittal process. Ideally,

the two participants should be based out of different organizations. In addition, while the SQA

evaluator and the sponsor can be collocated at the same site, they should be functionally

separated.

Before a software application and an independent evaluation are transmitted to DOE for

consideration to the Central Registry, it is recommended that the software sponsor notify DOE

Office of Quality Assurance Programs of its intentions. The notification will allow DOE to

review usage characteristics of the software and the credentials of the designated evaluator. A

screening review of this nature will minimize software and evaluation submittals that are not

likely to be successful.

B.2 PROCESS

B.2.1 ADDING SOFTWARE APPLICATIONS TO THE CENTRAL REGISTRY

Submittal of a software application for consideration as toolbox-equivalent is a two-phase

documentation effort, consisting of strategic benefits and SQA technical basis phases. In

principle, the first phase should be prepared by the software sponsor, and needs to establish the

basis or rationale for including the software in the Central Registry. At minimum, the discussion

in the first phase should establish the following.

• Widespread use of the software across the DOE complex for safety related applications.

• Methods to ensure proper software information, error reporting, configuration control and

other SQA management interface with the Central Registry.

• Demonstrated and quantifiable benefit for designating the software for the Central

Registry.

The second phase, the SQA technical basis phase, is initiated with completion of an independent

SQA evaluation, and is performed by the software evaluator. The evaluation should demonstrate

satisfactory compliance with toolbox-equivalency criteria and requirements established for the

DOE G 414.1-4 APPENDIX B

6-17-05 B-5

DOE

progr ct

software. An input template for this purpose has been developed, and is recommended as a

starting point mechanism to solicit b m the software sponsor. An

electronic copy may be obtained from SQA Knowledge Portal under the SQA Library (Software

), http://www.eh.doe.gov/sqa/doc_library.htm

Central Registry. The software sponsor is requested to provide information on the

ams and procedures associated with the development, maintenance, and use of the subje

asic SQA information fro

Information Template .

. Software design and related documents

action reports

anuals, and training packages/user qualification documents

red because they

compliance with the primary criteria. Furthermore, formal documents

in verifying completion of an action.

ssociated with

lso

Central Registry that best matches the DOE O 414.1C SQA work activity. For

many of the SQA work activities, the match is only partial, (i.e., not all of a specific work

practice is covered by the SQA toolbox requirement).

The input template seeks the following set of documents from the software developer.

1. Software project management and SQA plans

2. Software risk management documents

3. Software configuration management plan

4. Procurement and supplier management documents

5. Software requirements specifications

6. Software design, model description, programmer’s reference, and related documents

8. Verification and validation, test report, and other documents

9. Software error notification and corrective

10. User instructions, user m

Files, reports, telephone conferences, and other documented communications can provide

confirmatory indications that actions have been performed in SQA, and these can be used in lieu

of the availability of formal documents. However, formal documents are prefer

explicitly demonstrate

reduce the uncertainty

Software practices discussed in Section 5 of this Guide and the corresponding documents for

assessing compliance are listed in Table B-1, and are similar to those used in the evaluation of

he initial software applications designated for the Central Registry. The details at

each work activity are discussed in detail in the SQA plan and criteria document at the SQA

Central Registry Web site.

Because current and potential Central Registry software is best described under the custom

developed category, requirements for evaluation of software should be consistent with the

grading approach for custom developed software. Table B-2 lists SQA work activities discussed

in Section 5 of this Guide for custom developed software at both A and B grading levels. A

shown is the SQA requirement from those used to evaluate the initial software applications

designated for the

U.S. Department of Energy, Software Quality Assurance Plan and Criteria for the Safety Analysis Toolbox Codes,

Revision 1, dated 11-03.

APPENDIX B DOE G 414.1-4

B-6 6-17-05

Table B-1. Software Quality Assurance Work Activity and Corresponding

Documentation for Demonstrating Compliance

DOE O 414.1C SQA Work Activity SQA Documents

1. Software Project Management and

Quality Planning

- Software Project Management Pl

(SPMP) and/or

- Software Quality Assurance Plan

(SQAP)

- Software Safety Plan

2. Software Risk Management

- Various document types can be

used to cover risk management

3. Software Configuration Management - Software Configuration

Management Plan (SCMP) or

related documents

4. Procurement and Supplier Management - Contractual documents or other

Software procurement and use

agreement documentation

5. Software Requirements Identification

and Management

- Software Requirements

Specifications (SRS) or related

document

6. Software Design and Implementation

odel Description, Programmer’s

Reference Manual, or other related

- Software design description (SDD)

documents

7. Software Safety SDD -

- Software Safety Analysis

documentation

8. Verification and Validation - Verification and Validation Report

- Test Case Description and Outcome

Report; Other testing documents

9. Problem Reporting and Corrective

Action

- Software Error Notification and

Corrective Action Report

ftware

- Training Packages and User

Qualification

. Training of Personnel in the Design,

Development, Use and Evaluation of

Safety So

- User Instructions or User Manuals

DOE G 414.1-4 APPENDIX B

6-17-05 B-7

the

and

ation. Examples of

alternative information are previous reviews, older documentation from the code developer,

ne individual or a team of subject matter

valuation

size, those involved should be experienced

he evaluation of the software work activities

equately evaluate the constituent

f compliance was used with the designated toolbox

pliance conditions: Yes (meets requirement), No

completion of the evaluation of each of the

iew results as a whole and render an overall

a verifiable, objective

nner

B.2.1.1 Evaluation Process

The SQA evaluator performs and documents a review of the software, using the inputs from

code developer, including the responses in the Software Input Template or the equivalent,

other communications. In cases where the software developer is unable to supply requested

inputs, the SQA evaluation may consider alternative sources of inform

technical and journal articles, and previous software comparison studies.

The size of the actual SQA evaluation effort, whether o

experts, depends on the complexity of the software application. Regardless of SQA e

team in use of the software, but also knowledgeable

of the evaluation criteria. It is recommended that t

covered in Table B-1 use a sub-matrix of finer criteria to ad

parts of the requirement. Qualitative ranking o

software, applying the four terms defining com

(does not meet requirement), Uncertain (insufficient information available to evaluate), and

Partial (some but not all criteria are met). Upon

SQA work activities, the SQA evaluator can rev

assessment. The process leads to a firm basis to document findings in

ma .

Table B-3 contains a procedure for evaluating toolbox-equivalent candidate software, defined

the custom developed category for most safety applications. The overall evaluation process

shown schematically in Figure B-1. Input information for the evaluation is based on receipt of a

Software Information Template. An electronic copy may be obtained from SQA Knowledge

Portal under the SQA Library (Software Information Template),

http://www.eh.doe.gov/sqa/doc_library.htm.

While grading Level C cases can be postulated, it is believed that most software application

candidates for the Central Registry are categorized best under grading Levels A and B.

The SQA evaluation (gap analysis) reports performed on the six initial toolbox codes are a

reasonable level of detail for SQA evaluation documentation. While the SQA requirements and

criteria used for the toolbox codes are similar to those described in this Guide, they differ in

ped.

emphasis and extent of coverage. Thus, the gap analysis reports are illustrative, but not directly

applicable models. Instead, a software evaluation template for this purpose has been develo

The toolbox-equivalent software input and evaluation templates, as well as, copies of the gap

analysis reports and the full SQA evaluation plan and criteria document, can be downloaded

from the Central Registry Web site (http://www.eh.doe.gov/sqa/central_registry.htm

If previous reviews are used in whole or in part, it is required to confirm that the older review results are still

applicable.

APPENDIX B DOE G 414.1-4

B-8 6-17-05

Tab ng

Leve ities

le B-2. Software Quality Assurance (SQA) Requirements by Software Gradi

l and Matching DOE O 414.1C SQA Work Activ

Software Grading

Level

DOE O 414.1C

SQA Work Activity*

Level A

Custom

Level B

Custom

Corresponding SQA Toolbox

Software Requirement*

(a) Software project

management & quality planning

Full** Full 2. SQA Procedures and Plans

(b) Software risk management Full Grade ot addressed in the list of SQA d*** N

requirements.

manage

Configuration Control

ment

Full Full 12.

14. Access Control

(d) Procurement and supplier

manage

Full Fu

ment

ll 3. Dedication

(e) Software requirements

identification and management

FuFull ll 5. Requirements

(f) Software design and

implem

Full Full

entation;

6. Design

7. Implementation

(g) Software safety Full Gr dea d 6. Design

(h) Verification and validation Full Grade

est

enance

d 8. Testing

10. Acceptance T

11. Operation and Maint

(i) Problem reporting and

corrective action

Full Graded 13. Error Impact

(j) iTra ning of personnel in the

design, development, use and

evaluation of safety software

FuFull ll 9. User Instructions

*The SQA requirements used for evaluation of the initial set of s tions designated for the Central

Regi are m orresponding SQA work activi ro O 414.1C. See Table 3-3 of U.S.

Dep re Quality Assurance Plan an r the Safety Analysis Toolbox Codes,

Revision 1, (November 2003) for details on the requirement d e.

**Required for the computer software

*** d d based on judgm

oftware applica

stry atched to the c

artment of Energy, Softwa

ty f m DOE

d Criteria fo

s an the labeling (numbering) schem

Gra ed depending on the application an ent of SQA evaluator.

DOE G 414.1-4 APPENDIX B

6-17-05 B-9

Evaluation of Candidate Software for Central Registry Table B-3. Plan for

Step Procedure

1. Review Documentation

Determine that sufficient information is provided by the

software developer to allow proper classification of the softwa

Review developer reports, previous evaluations, and conference

and journal submittals, etc.

re.

Interview software developer.

2. Evaluate Justification

(Rationale) for Including

Software in Central

Registry

Review software sponsor’s document:

Widespread use of the software across DOE complex for safety

related applications?

Methods to ensure proper software information, error reporting,

configuration control and other SQA management interfaces

with the Central Registry?

Demonstrated and quantifiable benefit for designating the

software for the Central Registry?

3. Process Software

Information Template

Download template from Software Quality Assurance Web sit

http://www.eh.doe.gov/sqa/doc_library.htm

Confirm graded level determination.

4. Assess Software Project

Management and Software

Quality Assurance Plans

Review software project management plan (SPMP) and software

quality assurance plan (SQAP) for—

• required activities, documents, and deliverables and

• level and extent of reviews and approvals, including interna

and independent review.

Confirm that a

ctions and deliverables (as specified in the SQAP)

have been completed and are adequate.

uments;

• software error notification and corrective action reports; and

• user instructions, user manuals, and training packages/user

qualification documents.

Review engineering documentation identified in the SPMP an

SQAP, including—

• software risk management documents;

• software configuration management plan;

• procurement and supplier management documents;

• software requirements specifications;

• software design, model description, programmer’s reference,

and related documents;

• software design and related documents;

• verification and validation, test report, and other doc

APPENDIX B DOE G 414.1-4

B-10 6-17-05

Step Procedure

5. Assess SQA Work

Activity

Review aga

the Software Evaluation Tem late for DOE O 414.1C SQA

Work A

• software project management & quality planning,

• software risk m ment,

software configuration management,

procurement and supplier ma

• software requirements identi nagement,

• software design and implem

• software safety,

verification and validation,

• problem reporting and corrective action, and

• tr of pers in the d t, use and

evaluation of safety software.

SQA documentation inst detailed criteria found in

ctivities:

anage

•

• nagement,

fication and ma

entation,

•

aining onnel esign, developmen

6. Document Evaluation

Using Software

Evaluation Template.

Use g alysis re as examap an ports ples.

B.2.1.2 Submittal to the Central Registry

O s been c cted and nted be submitted to

t th wing ca

1 ncludes in the software evaluation (gap analysis) that software

ite criteria in the ten SQA work activities, and no criterion is

2 dentified one or more criteria not compliant for the subject

or can document a

compelling technical basis for submitting the software as “toolbox-equivalent” to the

Central Registry. Part of the technical basis should include a software application

guidance report that points out specific limitations and weaknesses and provides

instructions to the user on informed use of the subject software despite the identified gaps

and other vulnerabilities. Examples of guidance reports prepared for the initial six codes

designated for the Central Registry may be downloaded from the DOE SQA Web site at

http://www.eh.doe.gov/sqa/doc_library.htm

nce the SQA evaluation ha ondu docume , the software may

he Central Registry under one of

. The software sponsor co

has met all major requis

e follo ses.

evaluated as “No (=not met).” In other words, all significant improvement actions are

completed before the software is submitted for consideration as “toolbox-equivalent” t

the Central Registry.

. The software sponsor has i

software based on the gap analysis. However, the software spons

DOE G 414.1-4 APPENDIX B

6-17-05 B-11

If ith ftware

sponsor may move forward wi t

message should be sent to sqa@eh

Figur

all substantive issues in e

B-1. Flow Sheet for Software Evaluation

er Case 1 or Case 2 are satisfactorily dispositioned, the so

th he toolbox software submittal process. An electronic mail

.doe.gov, requesting a review of the evaluation and

designation of the software as

should be transmitted as attach

The DOE Office of Quality As

Table B-4 lists several of the k on to include the

candidate software in the Cent e candidate software

as a toolbox software applicati w oper and evaluator

organizations. If the decision i software

in question, and a general noti w eb site. Additional

notification methods may be im

Central Registry software coll io

If, on the other hand, issues w t n the software

sponsor is advised not to proce

examine continued use of the s tw

software, such as software curr

application.

a toolbox software application. All supporting documentation

ments.

surance Programs will review the submittal in a timely manner.

aey cceptance criteria for rendering a decisi

ral Registry. A decision on designation of th

on ill be communicated to the software devel

s favorable, the appropriate links will be provided for the

ce ill be posted on the Central Registry W

plemented to ensure broad notification of the changes in the

ect n.

ith he subject software are irreconcilable, the

ed further with the submittal process. It may be prudent to

of are at the site in question, and explore use of alternative

ently contained in the Central Registry, for the specific safety

1. Review Sw

umentation &

Doc

Interview Sw

Developer

• Software

development reports

• Previous Sw

evaluations

• Journal & conference

documents

2. Evaluate

Justification for

Including Sw (or

New Sw Version)

in Central

Registry

3. Process Software

Information

Template

(Includes graded level)

4. Assess

Software

Project

Management

and Software

Quality

Assurance

Plans

5. Assess Software Quality Assurance Work Activities

Software configuration management

ware design and implementation

d validation

(a) Software project management & quality planning

(b) Software risk management

(c)

(d) Procurement and supplier management

(e) Software requirements identification and

management

(f) Soft

(g) Software safety

(h) Verification an

(i) Problem reporting and corrective action

(j) Training of personnel in the design, development, use

and evaluation of safet

software

6. Document Outcome of Evaluation

Software Evaluation Report

r including

• Compliant areas

• Areas for improveme

• Overall assessment fo

software in Central R

• Determine whether su

Central Registry alon

eport

egistry

itable for

g with Code

Guidance R

APPENDIX B DOE G 414.1-4

B-12 6-17-05

Table B-4. Primary Criteria for Deciding on Inclusion of Software to the Central Registry

Phase Criterion*

1. Rationale for Adding

Software to Central

Registry

a. fety

ns.

b. reporting,

rance

entral Registry.

c. esignating the

Widespread use of the software across DOE complex for sa

related applicatio

Methods to ensure proper software information, error

r software quality assuconfiguration control, and othe

(SQA) management interfaces with the C

Demonstrated and quantifiable benefit for d

software to the Central Registry.

2. SQA Technical Basis a. ly demonstrates that

are has met all major requisite criteria, and

o (=not met).” If remedial tasks

it is

b. entified one or more

analysis. However, a compelling technical basis is made for

submitting the software as “toolbox-equivalent” to the Central

art of the technical basis should include a guidance

oints out specific limitations, weaknesses, and

The SQA evaluation document adequate

the candidate softw

n criterion is evaluated as “No

were cited before all criteria are considered met,

determined that these have been completed.

The SQA evaluation document has id

criteria not compliant for the subject software based on the gap

Registry. P

report that p

provides instructions to the user on informed use of the subject

software despite identified gaps and other vulnerabilities.

*Th artial list—others may be added as the process for software addition matures.

REVISIONS TO SOFTWARE APPLICATIONS IN THE CENTRAL REGISTR

ypical life-cycle processes associated with most software applications, updates,

ements, and modificat

is is a p

B.2.2 Y

In the t

improv ions will be made. Similar to software that is being considered for

irst tim r

The sam

outline

3. Upon conclusion of the evaluation and issuance of the SQA evaluation report (the gap

analysis), the software sponsor decides whether software has satisfactorily met all

requisite criteria for the ten SQA work activities, the revised software may be submitted

to the Central Registry.

the f e, revised software in the form of a new software version may also be submitted fo

inclusion in the Central Registry, with accompanying removal of the older version.

e process is followed for revised software to be placed in the Central Registry as is

d above for new software applications. The steps may be summarized as follows.

The software sponsor identifies the SQA evaluator organization.

The evaluator performs a complete evaluation over all aspects of the new software

version, emphasizing new and revised aspects of the software application.

DOE G 414.1-4 APPENDIX B

6-17-05 B-13

4. As noted earlier for new software applications to the Central Registry, an electronic mail

message should be sent to [email protected], requesting a review of the evaluation and

designation of the software as a toolbox software application. All supporting

documentation should be transmitted as attachments.

5. The DOE Office of Environment, Safety and Health, Office of Quality Assurance

Programs will review the submittal and decide on designation of the candidate software

as a replacement version to existing toolbox software. Upon reaching a favorable

determination, the appropriate links will be provided for the software version, and a

general notice will be posted on the Central Registry Web site regarding a new software

revision. In parallel with this action, the older software version will be removed from the

Central Registry and designated as an “archived toolbox version.”

.2.3 REMOVAL OF SOFTWARE APPLICATIONS FROM THE CENTRAL

REGISTRY

oftware applications are also subject to being removed from the Central Registry. Several

auses for this action include but are not limited to the following.

. The software developer indicates that older versions will no longer be supported and

elects to retire the software.

. New survey information indicates that few if any sites are using the software and that

other software applications

. The DOE Office of

the Web site for a comment period of 60 days, and no

revision, and removal of software or when it is necessary

ken

are being used for the specified safety applications.

Environment, Safety and Health, Office of Quality Assurance 3

Programs may make a decision to formally remove the software due to accumulated

evidence of unsatisfactory SQA events. Significant software errors in the subject software

or other factors may lead to this outcome.

Regardless of the basis, the subject software application may be removed from the Central

egistry after notification is posted onR

compelling evidence is received that conflicts with the planned removal action. The notification

should cite the basis or bases for the removal along with supporting documentation.

Upon reaching the end of comment period, the software application is then removed from the

Central Registry and designated as an “archived software application.”

B.2.4 ISSUE RESOLUTION AND ACTION COMMUNICATION

Actions will be taken on the addition,

to communicate information about software contained in the Central Registry. Several

communication mechanisms may be used to alert DOE staff, DOE software user, and

stakeholder groups. The extent of the communication will be commensurate to the action ta

or the importance to safety of the issue, and will be decided by the DOE Office of Environment,

Safety and Health, Office of Quality Assurance Programs.

APPENDIX B DOE G 414.1-4

B-14 6-17-05

more of the following.

1 vir

ffi

.doe.gov/ tm

Several of the primary mechanisms may include, but are not limited to, announcements to one or

. DOE Office of En

within the DOE O

(http://www.eh

onment, Safety and Health Central Registry Web site home page

ce of Environment, Safety and Health Knowledge Portal

sqa/central_registry.h

)

2. Safety Analysis Wor

Accident Analysis Subg

king Group in EFCOG, particularly its Steering Committee, and its

roup (http://www.efcog.org/workgroups/sawg-aa/

)

3. Radiation Safety Information Computational Center (RSICC), (http://www-

rsicc.ornl.gov/rsicc.html)

4 R

5. Formal Letter Notificati

1. American Society of nal

Nuclear Quality Assuran

Oversight, Letter to Lint

2. ASME NQA-1-2000, Qu s,

American Society of Me

3. ASME NQA-1A-1999, A

f Mechanical

ment of

del

assurance standards—Part 3: Guidelines for the application of

. Use of the Central egistry e-mail distribution list

on to the Program Secretarial Officers

B.3 REFERENCES

Mechanical Engineers, Re: Comments on the Benefits of Natio

ce Standards for NNSA and DOE Nuclear Activities and

on F. Brooks, NNSA (2002).

ality Assurance Requirements for Nuclear Facility Application

chanical Engineers, 2001.

ddenda to ASME NQA-1-1997, Quality Assurance

Requirements for Nuclear Facility Applications, American Society o

Engineers, 1999.

4. 10 CFR 830, Nuclear Safety Management.

5. 10 CFR 50, Appendix B, Quality Assurance Criteria for Nuclear Power Plants and Fuel

Reprocessing Plants.

6. DNFSB/TECH-25, Quality Assurance for Safety-Related Software at Depart

Energy Defense Nuclear Facilities, Defense Nuclear Facilities Safety Board Technical

Report, January 2000.

7. DNFSB Recommendation 2002-1, Quality Assurance for Safety-Related Software,

Defense Nuclear Facilities Safety Board, September 2002.

8. International Organization for Standardization (ISO)9001-1994, Quality Systems—Mo

for Quality Assurance in Design, Development, Production, Installation and Servicing,

ISO, 1994.

ISO 9001-2000, Quality Management Systems—Requirements, ISO 9000-3, ISO Quality

management and quality

DOE G 414.1-4 APPENDIX B

6-17-05 B-15 (and B-16)

10. afety and Health, Designation of

11.

ISO 9001:1994 to the development, supply, installation and maintenance of computer

software.

U.S. Department of Energy, Office of Environment, S

Initial Safety Analysis Toolbox Codes, Memorandum to Linton Brooks, Defense

Programs and Jessie Hill Roberson, Office of Environmental Management, March 28,

2003.

DOE-STD-3009-94, Preparation Guide for U.S. Department of Energy Nonreactor

Nuclear Facility Documented Safety Analyses, dated 7-94.

DOE G 414.1-4 APPENDIX C

6-17-05 C-1

APP

COMPLIANCE WITH DOE 10 CFR 830 SUBPART A AND DOE O 414.1C AND

SAFETY SOFTWARE

This ap

Program for Nuclear Facilities, and supporting standards for compliance with the Department of

Energy’s (DOE’s) quality assurance (QA) requirements (10 CFR 830 Subpart A and DOE

C.1. PURPOSE

This guidance may be used by organizations adopting ASME NQA-1-2000 as a national

consensus standard for development and implementation of a quality assurance program (QAP)

that meets the DOE QA requirements and includes safety software within its scope. This

appendix describes how ASME NQA-1-2000 addresses the DOE QA requirements and identifies

DOE QA requirements that are not addressed by ASME NQA-1-2000. Selected standards from

other standards bodies are included where emphasis or detail for safety software quality is

necessary.

C.2. INTRODUCTION

DOE QA requirements for activities that affect, or may affect, quality, nuclear safety or other site

specified criteria are established by 10 CFR Part 830 Subpart A, Quality Assurance

Requirements. DOE also has equivalent requirements for all other federal and contractor

activities in DOE O 414.1C. The DOE QA requirements and Guides are available for review at

http://tis.eh.doe.gov/nsps/quality.html.

The DOE’s objective of the QA Rule and Order is for organizations to establish effective

integrated management systems (i.e., QAPs) for the performance of DOE nuclear-related work.

The objective is accomplished through performance oriented QA criteria, coupled with

appropriate technical standards to manage, perform, and assess work activities. The DOE Rule

requires the use of voluntary consensus standards in the development and implementation of the

QAP. The ASME NQA-1-2000 standard is a national consensus standard, and as indicated in

DOE O 414.1C, ASME NQA-1-2000 or other national or international consensus standards that

provide an equivalent level of quality assurance requirements as NQA-1-2000 must be used for

providing the essential implementing methods for a QAP, including details for effective and

reliable supporting processes and procedures, as presented in this subpart.

C.3. DOE RULE AND ORDER GENERAL ADMINISTRATIVE QAP REQUIREMENTS

The DOE Rule and Order include both administrative and regulatory quality requirements. Those

administrative requirements relating to QAP approval authority, change control authority, and

compliance should not be relevant to the scope of ASME NQA-1-2000. Other administrative

quality related requirements that are relevant are addressed in Table C-1.

ENDIX C. USE OF ASME NQA-1-2000 AND SUPPORTING STANDARDS FOR

pendix provides guidance on the use of ASME NQA-1-2000, Quality Assurance

O 414.1C, Quality Assurance, dated 6-17-05) and their application to safety software.

APPENDIX C DOE G 414.1-4

C-2 6-17-05

C.4. DOE RULE AND ORDER QA CRITERIA

The DOE Rule and Order include ten QA criteria that are used to develop and implement a QAP.

Table C-2 identifies each of the ten DOE Rule and Order QA criterion and how they are

addressed by the ASME NQA-1-2000, Part I requirements. Differences in the documents and

topics that should be addressed independently of the ASME NQA-1-2000 criteria to meet the

DOE criteria are described. In some cases, the ASME NQA-1 Part II, “QA Requirements for

Nuclear Facility Applications,” and Part IV, “Non-mandatory Guidance in ASME

NQA-1-2000,” are also appropriate to address the DOE requirements and describe how the QA

criteria will be implemented. Table C-2 also includes selected standards from other standards

bodies (IEEE and IAEA) where they add emphasis or detail for safety software quality.

DOE G 414.1-4 Appendix C

6-17-05 C-3

TABLE C-1

10 CFR 830 S d uubpart A, date Jan ary 10, 2001

§830.121 a P Quality Assur nce rogram

DOE O 414.1C

DOE Genera ql Re uirements (Summarized) AS R iME NQA-1-2000 equ rements

Grad ( 7

Wher e,

appro m t

docu s h

subm e i

ion, nPart I, Introduct Requirement 1 a d Requirement 2

sing on activ

manner con

, however a

is applied an

ed Approach

e appropriat

ach to imple

ment the basi

it that docum

830.

a co

ent

of t

ntat

)

ntractor must use a graded

he requirements of this Part,

e graded approach used, and

on to DOE.

provid

r to y focu ities af

h plic nts in a sistent

m tanc ivity.

t does all proach DOE

describe roach d

to meet t nt.

t 3, 801.

es for

fecting

with

QAP

a graded app

quality and t

the relative i

The cited tex

will need to

documented

Requiremen

oach

e ap

por

achieving quality b

ation of requireme

e of the item or act

ow for a graded ap

how the graded app

he DOE requireme

4 provides grading

-2

rela

endix 2A

tive to software.

Part II, App Nonmandatory

idance on this topi

Guida ty A

cludes gu c.

, 101

nce on Quali ssurance

Programs in

Part IV, 4.1

nical Rep 97, ApIAEA Tech ort (TR) Series 3 pendix 1

QAP Development & Implementation

how the DOE QA criteria are The s

satisf

NQA-1 re ly m

t 2

The ASME

Requiremen

quirements partial eet the DOE requirement.

QAP must de

ied.

cribe requi ted QA d,

and mai es the for th

accompl es affe

t 5

res that a documen

ntained; and requir

ishment of activiti

P be planne

QAP provide

cting quality.

implemented

planning and

Requiremen

requi affect d serv

escrib ocum

procedur t inclu ce

ri uantitati ceptan r deter

es ed results ctorily atta

OE will need t b w the DOE crit are satisfied

res that “Activities

ed by and performed in acco

es, or drawings tha

ve or qualitative ac

have been satisfa

o descri e ho

ing quality an

rdance with d

de or referen

ce criteria fo

ined.”

eria

ices

ented

mining

should be pr

instructions,

approp ate q

that pr crib

A D QAP

Appendix C DOE G 414.1-4

C-4 6-17-05

TABLE C-1

10 CFR 830 Subpart A, dated January 10, 2001

§830.121 Quality Assurance Program

DOE O 414.1C

DOE General Requirements (Summarized) ASME NQA-1-2000 Requirements

Integrated Management Systems

The QA program must integrate the QA criteria with the

Safety Management System (SMS) or describe how the

QA criteria apply to the SMS.

The ASME NQA-1 requirements do not e

A DOE QAP will need to address integr r

address the DOE requirem

ation to meet the DOE crite

nt.

ion.

Ensuring Subcontractor & Supplier Quality

The QAP must describe how the contractor responsible

for the nuclear facility ensures that subcontractors and

suppliers satisfy the QA criteria.

Requirements 1, 2, 4, 7 and 18

The ASME NQA-1 requirements meet t

establishment of quality interfaces betwe

inclusion of applicable QA requirements ,

supplier evaluation activities and audits

A DOE QAP will need to describe how i

the DOE criteria.

he DOE requirement by the

en organizations, by the

in procurement documents

of suppliers.

subcontractors/suppliers sat sfy

DOE G 414.1-4 Appendix C

6-17-05 C-5

TABLE C-2

10 CFR 830 Subpart A, dated January 10, 2001

§830.122 Quality Assurance Criteria

DOE Quality Assurance

Criteria

ASM

Comments, Software

Requireme

E NQA-1-2000 Requirements

nts & Other

Standards

(documents noted in bold italic)

Criterion 1 -

Management/Program

NQA Requirements 1 and 2 Part IV, 4.1, 400

The ASME NQA-1 requirements meet the DOE criterion,

as noted.

IEEE 730-2002

IAEA Nuclear Safety Guide

IAEA TR 397, 2.2

(NS-G) NS-G-1.1, 4.11

(1) Establish an organizational

structure, functional

responsibilities, levels of

authority, and interfaces for those

managing, performing, and

The ASME NQA-1 r

the DOE criterion.

assessing work.

equirements satisfy this element of

None

(2) Establish management

processes, including planning,

NQA Requiremen

Basic meet the D

scheduling, and providing

resources for the work.

t 1,

senior management to

effective implementatio

and is responsible for

This implies that adeq

desired results.

to describe 201 General and Requirement 2, 100

criterion. ASME NQA-1 requires

establish overall expectations for

n of the quality assurance program

A DOE QAP will need

the management process for

providing resources.

obtaining the desired end result.

uate resources are provided to obtain

Appendix C DOE G 414.1-4

C-6 6-17-05

TABLE C-2

10 CFR 830 Subpart A, dated January 10, 2001

§830.122 Quality Assurance Criteria

Comments, Software

DOE Quality Assurance

Requirements & Other

ASME N irements QA-1-2000 Requ

Criteria

Standards

(documents noted in bold italic)

Criterion 2 -

Management/Personnel

Training and Qualification

NQA Requirement 2

The ASME NQA-1 requirements meet the DOE criterion.

(1) Train and qualify personnel to

be capable of performing their

The ASME

the DOE cr

assigned work.

(2) Provide continuing training to

personnel to maintain their job

proficiency.

NQA-1 r of

iterion.

DOE-STD-1172-2003 Safety

Software Quality Assurance

2.4

equirements satisfy these elements

Functional Area Qualification

Standard

IAEA TR 397,

IAEA NS-G-1.1, 4.9&10

Criterion 3 -

Management/Quality

Improvement

NQA Requirements 2, 15, and 16

The ASME NQA-1 requirements partially meet the DOE

criterion.

Part II, 2.7, 204

Part IV, 4.1, 204

IAEA TR 397, 2.5

A DOE QA Program will need to

extend the requirements of ASME

NQA-1 to ALL conditions adverse

to quality not just significant

conditions adverse to Quality.

TABLE C-2

DOE G 414.1-4 Appendix C

6-17-05 C-7

10 CFR 830 Subp anuary 10, 2001 art A, dated J

§830.122 Quality Assurance Criteria

Comments, Software

DOE Quality Assurance

Requirements & Other

Standards

ASME NQA-1-2000 Requirements

Criteria

(do lic) cuments noted in bold ita

(1) Establish and implement

etect and prevent

The ASME NQA-1 requirements partially meet the DOE

ming conditions from causing quality problems.

his is accomplished through various controls,

inspections, and tests. Requirement 16 includes criteria to

prevent recurrence of identified problems.

processes to d

quality problems.

criterion.

ASME NQA-1 provides a system of establishing quality

requirements and monitoring compliance to prevent

nonconfor

(2) Identify, control, and correc

items, services, and p

do not meet established

requirements.

rocesses that

The ASME NQA-1 requirements satisfy this element of

the DOE criterion.

(3) Identify the causes of

problems and work to prevent

ting

The ASME NQA-1 requirements partially satisfy this

element of the DOE criterion for “significant” or “generic”

recurrence as part of correc

the problem.

nonconformances.

(4) Review item characteris

process implem

tics,

entation, and other

quality-related information to

identify items, services, and

processes needing improvements.

The NQA requirements partially address this element of

the DOE criterion for known deficiencies.

Appendix C DOE G 414.1-4

C-8 6-17-05

TABLE C-2

10 CFR 830 Subpart A, dated January 10, 2001

§830.122 Quality Assurance Criteria

Comments, Software

DOE Quality Assurance

Requirements & Other

ASME NQA-1-2000 Requirements

Criteria

Standards

(documents noted in bold italic)

Criterion 4 -

Management/Documents and

Records

NQA Requirements 5, 6 and 17

The ASME NQA-1 requirements meet the DOE criterion.

(1) Prepare, review, approve,

establish design.

The ASME NQA-1 requirements satisfy these elements of

Part I, Requirement 3, 801

397, 2.6 & 3.1

issue, use, and revise documents

to prescribe processes, specify

requirements, or

(2) Specify, prepare, review,

approve, and maintain records.

the DOE criterion.

Part II, 2.7, 201 & 802

Part IV, 4.1, 201

IAEA TR

IEEE 730, 829

Criterion 5 - Performance/Work NQA Requirements 5, 8, 9, 12, 13, and 14 and the Part

I, Introduction

The ASME NQA-1 requirements meet the DOE criterion,

as noted.

Processes

(1) Perform work consistent with

technical standards, administrative

controls, and other hazard controls

adopted to meet regulatory or

contract requirements, using

approved instructions, procedures,

or other appropriate means.

The ASME NQA-1 requirements address “work” as

activities affecting quality.

(2) Identify and control items to

ensure their proper use.

The ASME NQA-1 requirements satisfy this element of

the DOE criterion.

will need to

Part I, Requirement 3, 802

Part II, 2.7, 203 & 404

A DOE QA program

address “work” as broadly as the

DOE criterion, since the

requirements for “work” are

derived from multiple sources in

the DOE Rule and Order.

TABLE C-2

DOE G 414.1-4 Appendix C

6-17-05 C-9

10 CFR 830 Subpart A, dated January 10, 2001

§830.122 Quality Assurance Criteria

Comments, Software

DOE Quality Assurance

Requirements & Other

ASME NQA-1-2000 Requirements

Criteria

Standards

(documents noted in bold italic)

(3) Maintain items to prevent

damage, loss, or deterioration.

their The ASME NQA-1 requirements satisfy this element of

the DOE criterion.

(4) Calibrate and maintain

equipment used for process

monitoring or data collection.

The ASME NQA-1 requirements satisfy this element of

Part IV, 4.1, 203 & 405

IAEA TR 397, 3.1 & 3.2

IEEE 828-1998 & 1219-1998

the DOE criterion.

Criterion 6 -

Performance/Design

NQA Requirement 3

The ASME NQA-1 requirements meet the DOE criterion.

(1) Design items and processes

applicable

asis in

approval and implementation of

the design.

ents of

Part II, 2.7, 401 & 402

Part IV, 4.1, 401 & 402

ANS-10.4

IAEA TR 397, 3.2 & 3.4

IEEE 1012-1998 & 1012A-1998

using sound engineering/scientifi

principles and appropriate

standards.

(2) Incorporate

requirements and design b

design work and design change

(3) Identify and control design

interfaces.

(4) Verify or validate the

adequacy of design products using

individuals or groups other than

those who performed the work

(5) Verify or validate work before

The ASME NQA-1 requirements satisfy these elem

the DOE criterion.

Appendix C DOE G 414.1-4

C-10 6-17-05

TABLE C-2

10 CFR 830 Subpart A, dated January 10, 2001

§830.122 Quality Assurance Criteria

Comments, Software

DOE Quality Assurance

Requirements & Other

ASME NQA-1-2000 Requirements

Criteria

Standards

(documents noted in bold italic)

Criterion 7 -

Performance/Procurement

NQA Requirements 4 and 7

The ASME NQA-1 requirements meet the DOE criterion.

(1) Procure items and services that

and

processes to ensure that approved

s and services.

The ASME NQA-1 requirements satisfy these elements of

Part II, 2.7, 300

meet established requirements

perform as specified.

(2) Evaluate and select

prospective suppliers on the basis

of specified criteria.

(3) Establish and implement

suppliers continue to provide

acceptable item

the DOE criterion.

Part IV, 4.1, 300

IAEA TR 397, 3.3

Criterion 8 -

Performance/Inspection and

Acceptance Testing

NQA Requirements 8, 10, 11, and 12

The ASME NQA-1 requirements meet the DOE criterion.

(1) Inspect and test specified

items, services, and processes

using established acceptance a

performance criteria.

(2) Calibrate and maintain

equipment used for inspections

ents satisfy these elements of

the DOE criterion.

IAEA TR 397, 3.4

IEEE 1008

and tests.

The ASME NQA-1 requirem

Part II, 2.7, 404

Part IV, 4.1, 404

ANS-10.4

TABLE C-2

DOE G 414.1-4 Appendix C

6-17-05 C-11

10 CFR 830 Subpart A, dated January 10, 2001

§830.122 Quality Assurance Criteria

Comments, Software

DOE Quality Assurance

Requirements & Other

ASME NQA-1-2000 Requirements

Criteria

Standards

(documents noted in bold italic)

Criterion 9 -

Assessment/Management

Assessment

NQA Requirement 2 and 18

The ASME NQA-1 requirements partially meet the DOE

criterion, as noted

Ensure managers assess their

management p

identify and correct pro

hinder the organization from

rocesses and

blems that

While ASME NQA-1, Requirement 2, 100 Basic, requires

y assess the adequacy and effective

nce, the DOE criterion is

Part II, 2.7, 202

art IV, 4.1, 202

IAEA TR 397, 4.1

of NQA

provide an input to this

will need

G 414.1-1A, Management

Assessment and Independent

Assessment Requirements of

10 CFR 830.120 and DOE O-

414.1, Quality Assurance, to meet

the DOE criterion.

achieving its objectives.

management to regularl

implementation of quality assura

broader in scope and intent.

IEEE 1028-1997

While audits per Req. 18

requirement, a DOE QAP

to align with the intent, focus, and

concepts described in DOE

Appendix C DOE G 414.1-4

C-12 6-17-05

TABLE C-2

10 CFR 830 Subpart A, dated January 10, 2001

§830.122 Quality Assurance Criteria

DOE Quality Assurance Criteria ASME NQA-1-2000 Requirements

Comments, Software

Requirements & Other

Standards

(documents noted in bold italic)

Criterion 10 - Assessment/Independent

Assessment

NQA Requirements 1, 2, 10, 11, 15, 16, and

The ASME NQA-1 requirements meet the

DOE criterion.

(1) Plan and conduct in

to measure item and service quali

dependent assessments

ty, to

easure the adequacy of work performance,

and freedom

group

sments.

m in

alifi

e areas to be assessed.

DOE defines assessment as a general term that

includes a variety of evaluation methods (i.e.;

reviewing, evaluating, inspecting, testing,

checking, surveillance, auditing, or otherwise

determining and documenting). As such,

several ASME NQA-1 requirements may be

necessary to address the various DOE

independent assessment methods. These

with the NQA

Assessment as a DOE activity for a

DOE QAP will need to align with

the intent, focus and concepts

described in DOE G-414.1-1A,

Management Assessment and

Independent Assessment

Requirements of 10 CFR 830.120

and DOE- O-414.1 Quality

Assurance.

and to promote improvemen

(2) Establish sufficient authority,

from line management, for the

performing independent asses

(3) Ensure persons who perfor

assessments are technically qu

knowledgeable in th

dependent

activities when combined

ed and

corrective action requirement have the intent

of the DOE criterion, to “promote

improvement.”

IAEA TR 397, 4.2

DOE G 414.1-4 APPENDIX D

6-17-05 D-1

APPENDIX D. QUALITY ASSURANCE STAN R SAFETY SOFTWARE IN

DEPARTMENT OF ENERGY NUCL ACILITI

D.1. INTRODUCTION AND R

The Department of Energy (DOE) stablishes

quality assurance requirements for n

may affect nuclear safety of DOE n

requirement that consensus standar

software is included in the scope o i d ensus

standards should be used for apply le and

consistent with contractual and regulatory requirements. This appendix describes practicable

standards for safety softw QA th used to satisfy the QA Rule.

D.2. REGULATORY AND QA PROGRAM COMPLIANCE

The ultimate responsibility for com th the QA Rule, and for selecting standards for

safety software that falls er the

contractor. Nuclear facil ontractors with DOE-approved QA Programs should ensure that any

changes to their QA Pro are m cordance with the QA Rule and any supplemental

DOE direction prov ugh co means.

D.3. QA PROGRAM STANDARDS VERSUS SOFTWARE STANDARDS

Dozens of consensu hav oped that address every aspect of software. In the

broadest sense of Q l se s ld be interpreted as “QA standards.” To develop

a useful report, it is lim n of standards to those that directly support

compliance with the DOE QA Rule and development of a QA Program that includes safety

software. There are other documents (e.g., technical reports, agency directives, and industry

guides) that may be useful as exam

developed through an accredited consensus standards process.

D.4. NDAR

Many of the standards loped a phases of software development rather than a

QA program that enco sses safe some cases the standards do cover a single

criterion within the QA program, such as training. Where this type of standard is used, it should

be in the context of r QA ncludes all criteria necessary for effective QA.

This report will diff etwe standards and standards that address a

specific criterion.

DARDS FO

EAR F ES

EGULATORY BASIS

ulation, 10 CFR 830 Subpart A, e

g providing items or services that affect or

lop and implement QA Programs. Safety

by the QA Rule. Therefore cons

oftware activities where practicab

nucl

activ

ucle

ds b

f act

ing Q

ear s

ities, inc

ar fa

e use

vitie

A t

afety

cilities. The Quality Assurance (QA) Rule includes a

d to

s co

o saf

reg

ludi

deve

vere

ety s

are

und

ity c

gram

thro

ndards

l of the

ssary to

at may be

plying wi

scope of the QA Rule, rests with the nuclear facility

ade in ac

ntractual

e been devel

tandards cou

it discussio

ples for application of the standards, but they are not

ided

s sta

A, a

nece

STA

deve

mpa

the broade

erentiate b

DS USE IN A QA PROGRAM CONTEXT

ddress specific

ty software. In

program that i

en QA program

APPENDIX D DOE G 414.1-4

D-2 6-17-05

D.5. QA P G AND SOFTWARE I A RD REQUIREMENTS

Identification of QA program standards for saf ider the following:

• co th the DOE QA Rule;

• rel lear facility safety

• applicability to software develope h ed;

• appli to entire software li y

• inclusion of monly accepted w a

. NA NA AN LEAR CILITY

Q LIT

Th omp nsive lear pro afety software is the

Am Soci f Me ical inee Assurance Program

for Nuclear Facilities. A dix f thi t are compatible

OE Q ule, be in ated rds, and is directly

to sa softw Mo portantly, ASME NQA-1-2000 expands upon the DOE

m re emen spe ally e quality, thus, placing

ware quality in con th fic software quality

• ME -1, P , Re e

• t II, Subpart 2 l r oftware for Nuclear

ility Applications; and

• Part IV, Subpart 4.1, Guide on Qu A ran equirem s for Software.

ASME NQA-1-2000 with Subpart 2.7 is also a practicable choi enting the DOE QA

Rule for safety software because it—

• is easily supplem d with other IAEA, IEC, a t

• provides independence for develo r

• support ded i mentation;

• is wide ed am DOE contra g

• is accredited as the American Nat d a on.

Table C-1 in A dix C Guide de A A ligns with DOE QA

criterion and includes other standards that n e E NQA-1

requirements for safety software.

mpatibility

evance to

cable

RAM

nuc

the

com

QUAL TY ST NDA

ety software should cons

ouse, purchased, or modifi

cle; and

ctivities for software QA.

;

d in-

fe-c

ork

D.6

rehe

ety o

A R

fety

quir

NQA

TIO

nuc

chan

ppen

can

are.

ts to

the

art I

.7, Qua

L ST

Eng

C o

tegr

st im

cific

text of

quirem

ity Assu

DARD FOR NUC FA

Y AND SOFTWARE

gram standard for application to s

rs ASME NQA-1-2000, Quality

s Guide includes SQA requirements tha

/supplemented with other standa

address requirements for softwar

e overall QA program. These speci

nt 3, Section 800, Design Control;

ance Requirements for Computer S

e m

eri

h th

lica

ety

uire

ost c

can

e D

ble

ogra

soft

ments are discussed in—

Par

Fac

wit

app

saf

req

ality ssu

pment and ve

ctor QA pro

ional Standar

scribes how

further expa

ce R

nd IEEE s

ification;

rams; and

for nucle

SME NQ

d the cont

ent

for implem

andards;

r applicati

-1-2000 a

nt of ASM

ente

mple

ong

of this

s gra

ly us

ppen

DOE G 414.1-4 APPENDIX D

6-17-05 D-3

Computers in Saf omputer

specific requirements addressing firmware, software, and hardware alike for the development

process in an integr f functional and

design requirements for computer components of a safety system employed in nuclear power

D.7. IN ARE

including software. The requirements and

guidance for nuclear facility quality are addressed in a 1996 Safety Series “Code” No. 50-C-Q,

Quality

Safety G

parallel the DOE QA Rule.

t to

IAEA

programs for safety software,

• Appendix

• Appendix III, considerations before acquisition of computerized tools;

nstitute of Electrical and Electronic Engineers (IEEE) Std 7-4.3.2-2003

, Criteria for Digital

ety Systems for Nuclear Power Generating Stations, describes c

ated approach. This standard recommends a minimum set o

generating stations. Additionally IEEE Std 1228, Standard for Software Safety Plans provides

requirements for the development of a management plan and performance of safety software

activities.

American Nuclear Society (ANS) standard, ANSI/ANS-10.4-1987

is supplemental to the IEEE

Std 7-4.3.2-2003 since it targets activities to improve the reliability of scientific and engineeri

computer applications while mitigating the risk of incorrect applications. Additionally IEEE h

a complete series of software and systems engineering standards to provide detail requirement

and guidance for the development of safety software.

TERNATIONAL STANDARDS FOR QUALITY AND SOFTW

D.7.1 INTERNATIONAL ATOMIC ENERGY AGENCY (IAEA)

The responsibility for international standards for nuclear safety is assigned to the International

Atomic Energy Agency (IAEA). The IAEA has a significant number of standards, guides, and

requirements for all aspects of nuclear facility safety,

Assurance for Safety in Nuclear Power Plants and other Nuclear Installations, and

uides 50-SG-Q1–Q14, respectively. The IAEA Code quality requirements closely

IAEA safety software guidance is detailed in Technical Reports (TR) Series No. 397, Quality

Assurance for Software Important to Safety. This TR provides information and guidance for

defining and implementing QA programs covering the entire life-cycle of software importan

safety. TR 397 was developed using a large amount of available information and standards an

offers implementation guidance that is tied to the QA program requirements found in the

Code. The application guides are useful aids for developing QA

specifically:

I, illustration of a graded software quality assurance program;

• Appendix IV, functions of computer program understanding and reverse engineering

tools;

IEEE Std 7-4.3.2-2003, IEEE Standard Criteria for Digital Computers in Safety Systems of Nuclear Power

Generating Stations, Institute of Electrical and Electronic Engineers, 2003.

American National S

for the Verification a

tandards Institute (ANSI)/American Nuclear Society (ANS) 10.4-1987 (R1998), Guidelines

nd Validation of Scientific and Engineering Computer Programs for the Nuclear Industry,

ANS, 1998.

APPENDIX D DOE G 414.1-4

D-4 6-17-05

•

Appendix VII, characteristics of defect prevention process;

ign input documentation for monitoring, control,

onitoring,

d procedures handbooks applicable to

• Appendix XII, recommendations on the content of software requirements specifications

for moni

• Appendix XIII, recomm iptions for monitoring,

control, and safety system software;

ndations on programming of monitoring, control, and safety

• ations on verification reports and activities for monitoring,

• Appendix XX, recommendations on commissioning monitoring, control, and safety

TR 397, and IAEA Safety Guide (SG) Series No. NS-G-1.1, Software for Computer Based

ormation that can be

e DOE QA Rule to produce an

ir relationship to the DOE QA Rule criteria and ASME

e for

f Nuclear Power Stations, IEC 987 Programmed Digital

Appendix V & VI, general training guideline and proposed outlines for training;

• Appendix VIII, examples of software development life-cycle models;

• Appendix IX, recommendations for des

and safety system software;

• Appendix X, recommendations for software development plans applicable to m

control, and safety system software;

• Appendix XI, recommendations for standards an

monitoring, control, and safety system software;

toring, control, and safety system software;

endations on software design descr

• Appendix XIV, recommendations on design and development documents for design,

engineering, and analysis software;

• Appendix XV, recommendations on application documents for design, engineering, and

analysis software;

• Appendix XVI, suggested good coding practices for design, engineering, and analysis

software;

• Appendix XVII, recomme

system software;

• Appendix XVIII, discussion of verification and validation methods;

Appendix XIX, recommend

control, and safety system software; and

system software.

Systems Important to Safety in Nuclear Power Plants, provide expanded inf

fully integrated with the ASME NQA-1-2000 requirements and th

effective quality program for safety software. Relevant portions of TR 397 are referenced in

Appendix C of this Guide to illustrate the

NQA-1 requirements.

D.7.2 INTERNATIONAL ELECTROTECHNICAL COMMISSION (IEC)

The IEC is responsible for several software standards in the nuclear power plant arena. These

standards are referenced in the IAEA TR 397. Those standards include IEC 880 Softwar

Computers in the Safety Systems o

DOE G 414.1-4 APPENDIX D

6-17-05 D-5

omputers Important to Safety for Nuclear Power Stations, IEC 1226 Nuclear Power Plants—

s and,

to maximize the reliability of the safety systems within a nuclear power plant.

O is not chartered to develop

standards for nuclear safety applications (this is the domain of the IAEA) and consequently lacks

sufficie that

face high hazards and high mission/political risk similar to DOE (e.g., aerospace, telecom,

O 9001 for application

AEA

d standards body for that subject, and (3) the IAEA offers safety

ftware quality standards compatible to DOE and ASME NQA-1, ISO should not be considered

g systems” of Level A applications. This standard recommends particular detailed

outputs for the software life-cycle processes, but is not prescriptive in how the outputs should be

Instrumentation and Control Systems Important for Safety—classification, IEC 61508 Parts 1-

Functional Safety of Electrical/Electronic/Programmable Electronic Safety Related System

IEC 61511 Parts 1-3 Functional Safety – Safety Instrumented Systems for the Process Industry

Sector.

The IEC Standard 880

is applicable to Level A highly reliable safety systems of nuclear power

plants. Like its Canadian counterpart, CE-1001-STD, IEC 880 standard advises various

approaches

D.7.3 INTERNATIONAL ORGANIZATION FOR STANDARDIZATION (ISO)

The International Organization for Standardization (ISO) is responsible for ISO 9001-2000,

Quality management systems – requirements. The ISO 9001 standard is designed for use

internally or as a contractual requirement for generic quality systems. ISO 9001 does not

specifically address computer software. More importantly, IS

nt focus (and rigor) to address DOE nuclear facility hazards. Commercial industries

chemical) have each issued supplemental requirements to improve on IS

to their industry.

Although ISO has a guide for applying a previous version of ISO 9001 (1994) to software

(ISO 9000-3, ISO Quality management and quality assurance standards—Part 3: Guidelines for

the application of ISO 9001:1994 to the development, supply, installation and maintenance of

computer software), this guide is not focused on nuclear safety.

Given that (1) the ISO standards are not developed for nuclear facility applications, (2) the I

is the internationally chartere

a practicable choice for standards in this subject area.

D.7.4 ATOMIC ENERGY CANADA LTD (AECL)

The Canadian standard

CE-1001-STD specifically recommends a minimum set of processes for

the software quality engineering of “safety critical systems used in real-time protective, control

and monitorin

obtained.

IEC 880, Software for Computers in the Safety Systems of Nuclear Power Stations, International Electrotechnical

Commission, 1986.

d Ontario Power Generation, Inc.,

Computer Society, 2000.

CE-1001-STD, Rev. 2, Standard for Software Engineering of Safety Critical Software, CANDU Computer

Systems Engineering Centre for Excellence, Atomic Energy of Canada Ltd. an

1999.

Herrmann, Debra S., Software Safety and Reliability: Techniques, Approaches, and Standards of Key Industrial

Sectors, IEEE

APPENDIX D DOE G 414.1-4

D-6 6-17-05

REQUIREMENTS AND PROCEDURES

hese

cume n, maintenance,

ampl

(VV&A ide. The guide also describes methods for assuring software

• been previously accredited based on verification and validation data which is available;

•

tion and validation available.

implem efinitions, roles and responsibilities for an

fety

Safety G sses to establish and implement

standards and procedures.

the DO ing 40 CFR 194. The 40 CFR 194 Rule and the

IPP Q E

comple

contracts for cleanup of certain Superfund sites. For these projects EPA has used the national

D.8. EXAMPLE APPLICATION GUIDES, FEDERAL AGENCY

D.8.1 DEPARTMENT OF DEFENSE (DOD)

The DoD software project requirement is Directive 5000.61 and related guidance. T

do nts address software development, verification, validation, accreditatio

review, and management. The documents also refer to national and industry standards. For

ex e, independent review is addressed in the Verification, Validation and Accreditation

) Recommended Practices Gu

using a graded approach depending on whether the software has—

been previously accredited based on historical use;

• not been previously accredited, but some verification and validation data available; or

not been previously accredited, with little or no verifica

DoD MIL-STD-882D

Appendix A is particularly useful because it supplies guidance for

enting a system safety effort, and the d

organization undertaking a new system safety effort. Similarly, the NASA Standard “Softwar

Sa NASA Technical Standard”

provides general guidance for a software safety effort.

D.8.2 NATIONAL AERONAUTICS AND SPACE ADMINISTRATION (NASA)

The NASA software document is the NASA 8739.8, Standard for Software Assurance. Also

reference NASA STD 8719.13B, Software Safety; and NASA-GB-8719.13 NASA Software

uidebook. The NASA standard includes proce

requirements and procedures, as well as, evaluating software products against requirements

D.8.3 ENVIRONMENTAL PROTECTION AGENCY (EPA)

The EPA uses at least two standards for software in environmental safety projects. EPA regulat

E Waste Isolation Pilot Project (WIPP) us

W A program requirements influence many other waste generation sites across the DO

x. The regulation adopts ASME NQA-1, 1989, and NQA-2A-1990 Part 2.7. EPA also

standard Quality Systems for Environmental Data and Technology Programs - Requirement

with Guidance for Use, ANSI/ASQC E4-1994. This standard is currently undergoing revision

and includes requirements for software quality that parallel ASME NQA-1-2000. The standard

also parallels the DOE QA Rule criterion.

DoD MIL-STD-882D, Standard Practice for System Safety, U.S. Department of Defense, dated 2-10-00.

NASA-STD-8719.13A, Software Safety, National Aeronautics and Space Administration, 1997.

DOE G 414.1-4 APPENDIX D

6-17-05 D-7

. PRACTICABLE STANDARDS FOR DOE QA RULE IMPLEMENTATION

The tables in Appendix C describe how ASME NQA-1-2000 aligns with DOE QA criterion. It

ents for

safety software to address appropriate elements for safety software quality.

are

A-1-2000, Quality Assurance Requirements for Nuclear Facility Applications,

in A Geologic Repository at

Yucca Mountain, Nevada.

ty Systems, Defense Nuclear Facilities

7. DOE Letter and Report, Quality Assurance for Safety-Related Software at Department of

D.8.4 DOE PROGRAM REQUIREMENTS AND PROCEDURES

The Department and its contractors have a variety of program requirement documents and

implementing procedures for safety software in use for nuclear facilities. However, the Yucca

Mountain Project’s quality assurance requirements document (QARD) DOE/RW-0333P has

been evaluated by an external regulatory body and found acceptable. The QARD and software

quality supplements describe a rigorous graded approach to safety software suitable for review

by other DOE organizations for use in developing their QA programs for safety software.

D.9

D.9.1 QA RULE & STANDARDS ALIGNMENT

also includes other standards that further expand the content of ASME NQA-1 requirem

D.9.2 STANDARDS LISTING

Appendix G of this Guide contains a listing of standards that may be applied to safety softw

to ensure quality.

D.10. REFERENCES

. ASME NQ1

American Society of Mechanical Engineers, 2001.

2. 10 CFR 830, Nuclear Safety Management.

3. 10 CFR 63, Disposal of High-Level Radioactive Wastes

4. DNFSB/TECH-25, Quality Assurance for Safety-Related Software at Department of

Energy Defense Nuclear Facilities, Defense Nuclear Facilities Safety Board Technical

Report, January 2000.

5. DNFSB/TECH-31 Engineering Quality into Safe

Safety Board Technical Report, March 2001.

6. DNFSB Recommendation 2002-1, Quality Assurance for Safety-Related Software,

Defense Nuclear Facilities Safety Board, September 2002.

Energy Defense Nuclear Facilities, DOE Response to TECH-25, U.S. Department of

Energy, October 2000.

APPENDIX D DOE G 414.1-4

D-8 6-17-05

ommendation 2002-1: Quality Assurance for Safety Software at Department of

Energy Nuclear Facilities, U.S. Department of Energy, March 13, 2003.

8. DOE Report, Implementation Plan for Defense Nuclear Facilities Safety Board

Rec

DOE G 414.1-4 APPENDIX E

6-17-05 E-1

APPENDIX E. SAFETY SOFTWARE ANALYSIS AND MANAGEMENT PROCESS

The following diagrams provide a recommended process flow for the analysis and management

of safety software applications.

Note:

everything

above dotted

line must be

done per

DOE-STD

3009-94

Apply Level A grading

practices

Develop documented

safety analysis in

accordance with DOE-

STD-3009-94 and DOE-

STD-3011-94

Graded safety

application–

using DO

O 414.1

Apply Level B grading

practices

Apply Level C grading

practices

Safety software

analysis process

Elicit & analyze

software

requirements

Review

documented

report)

safety analysis

(safety analysis

Typical

organizational

software policies

apply

Does the software

meet the definition

of safet

software?

Does the software

meet the definition

of safety software?

APPENDIX E DOE G 414.1-4

E-2 6-17-05

Are performance or quality

requirements required for

safety software application?

Define performance

and/or quality

requirements

Are data structures or database

design important for safety

software application?

Ensure data integrity with fault

tolerant data designs

Is safety software task an

upgrade to existing software?

Assess impact of new maintenance

safety software requirements upon

existing application

DOE G 414.1-4 APPENDIX E

6-17-05 E-3 (and E-4)

Does a product safet

verification and validation

(V&V) strategy exist?

Does the strategy conform to

Level A (or B

e) software?

DOE G 414.1-4

or C if appropriat

Verif

softw

y and validat

are functionality

e safety

Does software conform

to requisite safety

goals?

Develop safety

software V&V

strategy

Notify software developer of safet

software deficiencies for corrective

action.

Determine required

safety software

quality assurance

activities and

document in gap

analysis

Place base

and softwa

configuratio

line documents

re under

n control.

Safety software is

ready for use

End safety

software

analysis process

Develop strategy

to eliminate deltas

noted in gap

analysis

DOE G 414.1-4 APPENDIX F

6-17-05 F-1 (and F-2)

DOE G 414.1-4 APPENDIX F

6-17-05 F-1 (and F-2)

F-3

F-4

F-7

F-8

F-9

F-10

F-14

F-18

APPENDIX F. DOE O 414.1C CRITERIA REVIEW AND APPROACH DOCUMENT

CONTENTS

F.1 INTRODUCTION ...........................................................................................................

F.2 PURPOSE AND SCOPE.................................................................................................

F.3. GUIDING PRINCIPLES.................................................................................................

F.4. ASSESSMENT METHODOLOGY................................................................................

F.5. CRITERIA AND APPROACH.......................................................................................

F.5.1 Software Project Management and Quality planning....................................

F.5.2 Software Risk Management...........................................................................

F.5.3 Software Configuration Management..........................................................

F.5.4 Software Procurement and Supplier Management ......................................F-11

F.5.5 Software Requirements Identification and Management.............................F-12

F.5.6 Software Design and Implementation..........................................................F-13

F.5.7 Software Safety............................................................................................

F.5.8 Software Verification and Validation..........................................................F-15

F.5.9 Software Problem Reporting and Corrective Action...................................F-16

F.5.10 Training Personnel in the Design, Development, Use, and

Evaluation of Safety Software .....................................................................F-17

F.6. REPORT FORMAT.......................................................................................................

DOE G 414.1-4 APPENDIX F

6-17-05 F-3

DOE O 414.1C CRITERIA REVIEW AND APPROACH DOCUMENT

This document contains software qualific t criteria and guidelines for assessing

the safety system software used in Department of Energy (DOE) defense nuclear facilities.

•

oftware includes safety analysis; design of structures, systems, and components (SSCs); human-

machine interface software; network interface software; programmable logic controller (PLC)

programming language software; and safety management software in DOE defense nuclear

facilities. DOE O 414.1C includes the definitions for safety software.

The assessment criteria and guidelines ensure that the software being used in DOE’s nuclear

facilities is adequate. The primary set of baseline SQA criteria for evaluating safety software are

based on the following:

• 10 CFR 830, Nuclear Safety Management;

• DOE O 414.1C, Quality Assurance, dated 6-17-05;

• American Society of Mechanical Engineers (ASME) NQA-1-2000, Quality Assurance

Program for Nuclear Facilities; and

• applicable Institute of Electrical and Electronics Engineers (IEEE) standards.

F.1 INTRODUCTION

ation assessmen

This document is organized as follows.

• The Assessment Guidelines section covers the purpose, scope, guiding principles, and

assessment methodology for assessing the processes currently in use for ensuring the

adequacy of safety software.

• The Criteria and Approach section presents the objective, criteria, approach, and

tailoring for the following work activities: (1) Software Project Management and Quality

Planning, (2) Software Risk Management, (3) Software Configuration Management

(SCM), (4) Software Procurement and Supplier Management, (5) Software Requirements

Identification and Management, (6) Software Design and Implementation, (7) Software

Safety, (8) Software Verification and Validation, (9) Software Problem Reporting and

Corrective Action, (10) Training of Personnel in the Design, Development, Use, and

Evaluation of Safety Software.

The Report Format section provides a suggested report format.

The References section lists selected references relevant to software quality assurance

(SQA)

F.2 PURPOSE AND SCOPE

The purpose and scope of the criteria review and approach document (CRAD) is to provide a set

of consistent criteria and guidelines for the assessment of the safety software. The safety

APPENDIX F DOE G 414.1-4

F-4 6-17-05

The CRAD is not intended to be prescriptive enough to evaluate the overall QA program

associated with the safety software but rather to focus on the safety software application/product.

Individual sites should tailor the scope of the assessment to suit the specific usage software in

their safety systems. The CRAD could be used for assessment of the following types of software:

• custom software developed by DOE, its contractors, or subcontractors for use with safety

systems or safety class (SC) SSCs;

• configurable, such as PLCs;

• acquired software, including commercial off-the-shelf (COTS) software;

• utility calculation software, such as spreadsheets and math programs (along with their

associated user files), used to perform safety analysis and design calculations; and

• commercial design and analysis tools, such as proprietary facility design and accident

analysis software.

These software types are used in the following safety software applications:

• safety management and administrative database programs and associated user files to

maintain control of information that has nuclear safety implications;

• instrumentation and control software, including embedded microprocessors, distributed

control systems, supervisory control and data acquisition systems, PLCs, and other

related software;

• networking and interface applications;

• safety accident analysis; and

• design and analysis of SSCs.

Should an issue arise that casts doubt on the validity of software previously used to support

design or development, it should be resolved using the unreviewed safety question (USQ)

determination (USQD) process. Generic USQs will be used to the extent possible to preclude

multiple facilities’ developing separate USQDs for the same issue.

F.3. GUIDING PRINCIPLES

The following principles should guide the conduct of the assessment. The assessment team

leader, with assistance from the DOE site manager responsible for these assessments, should

ensure that these guiding principles are incorporated in the tailoring process for assessing safety

software applications.

• The team should review any previous assessments and reviews of safety software. This

review will enable the team to understand previous assessments, software qualification

processes, associated requirements and performance criteria, assumptions concerning

system operations, and the role of safety software in operations.

DOE G 414.1-4 APPENDIX F

6-17-05 F-5

• Th tware

applications and include any propriate in the assessment plan.

• The review of SQA processes for existing safety software should follow the guidance

uld be agreed upon by DOE, the contractor line

r to the start of the assessment and should

•

entation, against the demonstrated effect on

result

•

of failure of the software and the associated effects on

documented evidence to the team that the appropriate SQA standards were

applied to software development, procurement, or use; and provide a staff point of

contact for further information.

• Procedures and records for software design, implementation, procurement, verification

personnel should

ill be necessary to tailor the assessment criteria and guidelines to focus

rmined to be appropriate for the assessment

nsure that the assessments are conducted in

ppropriate and applicable to each

re and its ability to operate safely on a continued

ntly

e team should review any lessons learned from past events associated with sof

additional attributes as ap

provided in the DOE G 414.1-4 Section 3.3.2 Existing Safety Software Applications.

The physical boundaries of the software within the safety system or subsystem level o

•

portions thereof under review sho

management, and the assessment team lead prio

be documented in the assessment report.

Care should be taken to balance the effort invested during the assessment in verifying the

SQA processes and their supporting docum

improving the software quality and safety, and on eliminating the costly errors that

from misunderstood requirements.

The assessment of specific software applications should begin with gaining an

understanding of the overall system, and documenting the system safety functions, the

performance criteria that the system should meet to successfully accomplish its safety

functions, and the role of the software in ensuring that these functions and criteria are

met. The potential consequences

system operability should be understood and documented.

• The facility staff should assist the team in understanding the associated SQA process;

provide

and validation (V&V), testing, and maintenance should be evaluated for adequacy and to

determine whether they are appropriate and are being used to verify that software

requirements and performance criteria described in the software requirements

documentation are satisfied.

• If the team identifies a condition that poses an imminent threat to personnel or facilit

safety, line management should be notified immediately. Team

immediately point out the imminent threat condition to their points of contact or

appropriate facility manager and notify the assessment team leader as soon as practical.

• In some cases it w

the assessment to address those aspects dete

scope. The tailoring process is intended to e

accordance with the criteria and guidelines that are a

specific situation. These assessment criteria and guidelines are intended to be flexible,

and may be tailored to allow the most cost-effective use of resources in determining the

operational readiness of safety softwa

basis. The tailoring process may take into account considerations, such as rece

APPENDIX F DOE G 414.1-4

F-6 6-17-05

. The

tailoring and its associated rationale should be agreed upon prior to the start of the

level of modification to the software when evaluating the

es. Acquired software, such as COTS, may not be modified

software is

tances

as custom developed software for the specific application. The

grading approach in this Guide assists in this effort.

• The assessment should consider the effectiveness of SQA processes that are separate

rease the safe

• hould

r any of the documentation, such as a problem statement, requirements

lan, or test results, is available. In situations

al documents do not exist, sufficient information may be

entation.

• used in analysis or design for several years,

iew

pment

d responses. The level of a

posteriori review may range from a simple demonstration that the software produces

reasonable results for a representative sample of inputs or test cases, to a rigorous

verification of program r coverage, and evaluation of test

results. The team may consider using documented engineering judgments (including their

ot be found, the assessment may consist primarily of a review of system

completed assessments, evaluations, studies, inspections, and other relevant factors

assessment criteria and guidelines in this CRAD are provided as a tool for use in

developing specific criteria and guidelines. It is recognized that some of the criteria may

not apply. This should be noted in the assessment report. For each assessment, the

assessment, and documented in the assessment report.

• The team should consider the

adequacy of the SQA process

and can be viewed within the system as a “black box.” Custom developed

completely modifiable and may require additional SQA processes over those of acquire

software. Some acquired software can be configured specifically for its application or i

source code can be modified to meet application specific requirements. In these ins

a higher level of SQA requirements should be expected. However, these requirements

may not be as high

from system quality processes. In many instances, especially with acquired software, th

separation of software from the system may increase costs but not inc

operation of the system.

Information for existing software may not be appropriately documented. The team s

determine whethe

specification, design specification, test p

where clearly identifiable form

contained in the system docum

For safety software that is in operations or

the assessment team should consider using an approach similar to the a posteriori rev

described in ANS 10.4. This approach takes advantage of available program develo

products and program staff, as well as, the experience of the users. The purpose of an a

posteriori review is to determine if the system produces vali

equirements, design, coding, test

bases) and test results to extrapolate the available existing information to establish

functional and performance capabilities.

• Using the a posteriori approach for existing software where some documentation does

not exist or cann

test procedures, test records, and verification process to ensure the test results are

consistent with the software requirements. Documentation of the software requirements

necessary to ensure that future changes to the software are adequately controlled and

consistent with system operation as assumed in the facility safety basis.

DOE G 414.1-4 APPENDIX F

6-17-05 F-7

a sessm e

system being assessed. The guidance for assessm

major a

•

fety software, which includes software that performs

a safety function as part of an SS and SC system as defined in the facility documented

•

o supplement and complement the documented

The Criteria and Approach section is divided into the following work activities.

F.4. ASSESSMENT METHODOLOGY

The assessment planning is to ensure assessments efficiently address the objectives of the

s ent. The level of planning will vary depending on scope and complexity of the softwar

ent planning is available in other DOE Guides.

In addition, for safety software assessments, the review team should consider the following

ctivities.

The team should prepare the assessment plan using the CRAD, and develop a question

set with lines of inquiry and detailed attributes, as appropriate, for site-specific

applications. The plan should include qualification requirements for team members, a

listing of team members and their biographies, a plan for the preassessment visit, and

guidance for preparing the report.

The CRAD is prepared to address sa

safety analysis (DSA) and technical safety requirements (TSRs). Safety software is an

integral part of a safety system. Safety software classification should be consistent with

SSC classification unless otherwise justified for case-specific application. The team

should use facility-specific DSAs and TSRs for the selection of safety software.

The team should review DOE O 414.1C, Quality Assurance, dated 6-17-05; this Guide;

NQA-1-2000; and other applicable standards for assistance in developing the lines of

inquiry and to determine their appropriateness for the safety software being assessed.

Appendix G of this Guide includes additional industry standards and guidelines.

The team should use interview methods, as well as, informal discussions with progr

developers, users, and sponsors t

information.

F.5. CRITERIA AND APPROACH

1. Software Project Management and Quality Planning

Software Risk Management

SCM

4. Software Procurement and Supplier Management

Software Requirements Identification and Management

Software Design and Implementation

414.1-1A, Management Assessment and Independent Assessment Guide for Use with 10 CFR, Part 830,

t A, and DOE O 414.1A, Quality Assurance; DOE P 450.4, Safety Management System Policy; and DOE

, Line ES&H Oversight Policy, dated 5-31-01.

DOE G

Subpar

P 450.5

APPENDIX F DOE G 414.1-4

F-8 6-17-05

10.

Each of these work activities includes the following.

•

Existin ftware may satisfy some of the

evan

assessm

should

A variety of software engineering methods may exist at DOE sites to meet the applicable SQA

associa

safety o of

custom

For each of the ten work activities, the SQA standards and guidance being applied by the

judgme

their im

F.5.1 S

Object

Softwa

that sup

quality

ia:

rming,

Software Safety

Software Verification and Validation

Software Problem Reporting and Corrective Action

Training of Personnel in the Design, Development, Use, and Evaluation of Safety

Software

Objective: Describes the assessment objective for the work activity and the intended

contribution to the adequacy of safety software.

Criteria: Suggests characteristics of safety software that should be verified.

Approach: Suggests information needed to guide the team in assessing the quality of the

safety software. However, the team may choose to select another approach to meet the

assessment-specific needs.

g QA or other requirements (e.g. procurement) for so

objectives and criteria for safety software. Previous reviews may also contain information

rel t to this assessment that can be cited and used in this assessment. In such situations, this

ent should be limited to objectives and criteria not covered in previous assessments and

not unnecessarily duplicate previous assessments.

requirements and work activities. These requirements should be commensurate with the risk

ted with a software failure. Factors affecting this risk include the potential impact on

r operation, complexity of computer program design, degree of standardization, level

ization, state of the art, and comparison with previously proven computer programs.

contractor should be documented in the assessment report along with the assessment team’s

nt of their appropriateness for the specific software application, and the effectiveness of

plementation.

OFTWARE PROJECT MANAGEMENT AND QUALITY PLANNING

ive:

re project management and quality planning should depict the organizational structure

ports the software life-cycle stages and deliverables, and influences and controls the

of the software.

Criter

Software project management and quality planning has been implemented depicting

organizational structure, responsibilities, and authorities for those managing, perfo

and assessing the software projects.

DOE G 414.1-4 APPENDIX F

6-17-05 F-9

2. sessed.

3. Software quality activities have been effectively implemented.

Approach:

Confirm the existence of project management and QA planning work activity. This may be

• software project scope;

•

;

•

Many o V

may be this

work ac tivity

being p relates to the grading level.

Determine whether the documents containing the software project management and quality plan

are controlled under configur ol process, and are

maintained until the software is retired. This may overlap with the SCM work activity.

dated, as

ap with the software V&V work

ivity

.2 S ARE RISK MANAGEMENT

Objective:

SQA activities, software practices, and documentation are periodically as

present in software project management and/or SQA plans that exist either as a stand alone

document or embedded in other documents and related procedures. The software project

management and software quality planning should identify and/or define the following:

• software project schedule;

• software engineering activities, including software requirements and design;

software V&V activities, including reviews and test;

SCM activities;

software risk management approach

• software safety analysis and planning;

supplier control;

user and software staff training,

standards, practices, conventions, and metrics;

• records and document collection, maintenance, and retention; and

problem reporting and corrective action methods.

f the items listed above may be detailed in other documents, for instance software V&

detailed in a software V&V plan or in software test plans. It should be noted that

work activity addresses the existence that these items are identified and described. Associated

tivities, such as software V&V address the quality of the software V&V work ac

erformed as it

ation change control and document contr

Verify that the software project management and quality plan is reviewed and up

necessary, for completeness and consistency. This may overl

act .

F.5 OFTW

oftware risk management is a proactive and disciplined approach to assess and control software

isks.

APPENDIX F DOE G 414.1-4

F-10 6-17-05

iteri

rading level.

fety software failure are determined.

ed.

es are created.

Appro

planning. This may be evident in a

manage

• identification of the technical and managerial risks and likelihood and potential safety

, including tracking,

2. A baseline labeling system is established and implemented.

dition, for Level A or Level B custom developed safety software, periodic

5. Only approved changes are implemented.

Cr a:

1. Potential software risks are identified as required by the g

2. Likelihood and consequences of the sa

3. Risks are prioritiz

4. Risk avoidance, mitigation, and/or transfer strategi

5. Risks are monitored.

ach:

Determine the existence of software risk management

standalone document or embedded in another document. As applicable, ensure that the risk

ment planning specifies the following:

• scope of the risk management activities;

• risk management policies and process (for both technical and managerial) under which

risk management is to be performed are defined;

consequences of using software risk taxonomies as a guide;

• establishment of risk thresholds for the safety software application;

• risk avoidance, mitigation, or transfer options; and

• management techniques to address risks throughout project life-cycle

decision, and feedback points.

F.5.3 SOFTWARE CONFIGURATION MANAGEMENT

Objective:

Software configuration is defined, maintained, and controlled until the software is retired.

Criteria:

1. Software configuration items are identified, baselined and controlled.

3. In ad

configuration audits and reviews are conducted and documented.

4. Proposed software changes are documented, evaluated, and approved.

DOE G 414.1-4 APPENDIX F

6-17-05 F-11

control

on the f

be, and

oftware and its related

documentation. This documented evidence may be in either SCM plan or embedded in

er software or system level document.

ed.

s’

ware

design, software V&V procedures, test plans and procedures, and any software

anning documents.

g system has been created that uniquely identifies each

provides

r installing new versions of the

ents, including new releases of acquired software.

ackages to ensure that (1) possible impacts

before changes are made, (2) various software

stency and revised as necessary after changes are

, (3) software is tested according to established standards after changes

evaluated and approved for release by the responsible

ed as necessary to ensure that the

ce of the software.

s accurately reflects

e software.

ify audits or reviews, such as functional

configuration audit or physical configuration audit, have been performed.

nts.

Approach

Review appropriate documents, such as applicable procedures related to safety software chan

to determine if an SCM process exists and is effective. This determination is made based

ollowing actions.

• Verify the existence of documented processes to control, uniquely identify, descri

document the configuration of each version or update of safety s

anoth

• Verify that a configuration baseline is defined and that it is being adequately controll

This baseline should include operating system components, any associated runtime

libraries, acquired software executables, custom-developed source code files, user

documentation, the appropriate documents containing software requirements, soft

development and quality pl

• Verify a baseline labelin

configuration item, identifies changes to configuration items by revision, and

the ability to uniquely identify each configuration.

• Review procedures governing change management fo

software compon

• Review software change packages and work p

of software modifications are evaluated

system products are examined for consi

made and updated

have been made, (4) changes are

organization, and (5) software validation is perform

change does not adversely affect the performan

• Verify by sampling that documentation affected by software change

all safety-related changes that have been made to th

• Interview a sample of cognizant line, engineering, and QA managers, and other personne

to verify their understanding of the change control process and commitment to manage

changes affecting design, safety basis, and software changes in a formal, disciplined, an

auditable manner.

• For custom developed safety software, ver

F.5.4 SOFTWARE PROCUREMENT AND SUPPLIER MANAGEMENT

Objective:

Acquired safety software, either COTS software or custom-developed for DOE, meets the

appropriate level of QA based on risk, safety, facility life-cycle, complexity, and project qualit

requireme

APPENDIX F DOE G 414.1-4

F-12 6-17-05

urement documents identify the technical and quality requirements.

Suppliers’ QA programs meet or exceed the QA requirements specified in the

procurement documents.

curement documents specify supplier reporting of software defects to the purchaser

plie e are evaluated to ensure that the safety software is developed

The assessment of

• Determine the existence of safety software technical and QA requirements. These

rements may be embedded in the DOE contractors’ or subcontractors’ procurement

document, software or system design description, or SQA plan. If not documented in the

s been reviewed and meets or exceeds the

e supplier may review the supplier’s QA

est summary, supplier site

may overlap with

e supplier and purchaser for a

eview may overlap with the Problem Reporting and

Corrective Action work activity.

ANAGEMENT

ged

throughout the safety software life-cycle.

Criteria:

1. Proc

2. Acquired software meets the technical and quality requirements.

4. Pro

and the purchaser’s reporting of defects to the supplier.

Approach:

Sup rs of acquired softwar

under an appropriate QA program and satisfies the specific requirements.

software procurement process should include the following.

requi

procurement contract, ensure that the supplier has received such technical and QA

requirements. This verification may overlap with the Software Requirements

Management work activity.

• Verify that the suppliers’ QA program ha

procurement specification requirements. Th

program through supplier assessment, supplier self-declaration, third-party certification,

or other similar methods.

• Review evidence that the acquired software was evaluated for the appropriate level of

quality. This evidence may be included in the test results, a t

visit reports or supplier QA program assessment reports. This review

the V&V work activity.

• Review procurement or other documents between th

documented process to report software defects from the supplier to the purchaser and the

purchaser to the supplier. This r

F.5.5 SOFTWARE REQUIREMENTS IDENTIFICATION AND M

Objective:

Safety software functions, requirements, and their bases are defined, documented and mana

1. The software requirements are documented and consistent with the system safety basis.

DOE G 414.1-4 APPENDIX F

6-17-05 F-13

rface

ct, consistent, clear,

controlled and maintained. Changes to the

and all documentation.

requirement should be uniquely identified and defined such that it can be

objectively verified and validated.

Review appropriate safety basis documents, such as DSAs, safety analysis reports, TSRs,

require re

require

specific re level documents.

• y

•

ted as necessary. This

Object

The safety software design depicting the logical structure, information flow, logical processing

plem

2. The functionality, performance, security including user access requirements, inte

and safety requirements for the safety software are complete, corre

testable, and feasible.

3. The documented software requirements are

software requirements are reflected in any

4. Each

Approach:

procurement specifications and any system documentation to determine if the safety software

ments document is consistent with the safety system design and safety basis. The softwa

ments may exist either as a standalone document, such as a software requirements

ation, or embedded in other system or softwa

Determine whether the following types of requirements are addressed as appropriate.

Verify that the software requirements address functionality, performance, security, safet

design inputs, design constraints, installation considerations, operating systems (if

applicable), and external interfaces necessary to design the software exist and ar

documented.

• If access to the system by only authorized users is a requirement, verify that use of

software is controlled so that only personnel on authorized user lists apply or maintain

safety software.

• Verify that the software requirements are correct, unambiguous, complete, consistent,

verifiable, modifiable and traceable as appropriate.

• Verify that acceptance criteria are established in the software requirements for each of the

identified requirements. Such criteria should be used for V&V planning and performanc

as defined in each related life-cycle phase.

Verify that the software requirement documents are controlled under the configuration

change control and document control processes. This may overlap with the SCM activity.

Verify that software requirement documents are reviewed and upda

may overlap with the software V&V work activity.

F.5.6 SOFTWARE DESIGN AND IMPLEMENTATION

ive:

steps, data structures and interfaces are defined and documented. The design is properly

im ented in the safety software.

APPENDIX F DOE G 414.1-4

F-14 6-17-05

esign, including interfaces and data structures, is correct, consistent, clearly

ign requirements are traceable throughout the software life-cycle.

and source code

ents.

• iption should contain the following information.

onsafety components.

echnical description of the software with respect to control flow, control logic,

thematical model, data structure and integrity, and interface.

puts

se of interrupt protocols.

•

Object

The des components are developed in a manner that ensures the

design

Criteri

s are

2. implicity and isolation of safety functions.

3. Where appropriate fault tolerance and self-diagnostics are implemented in the safety

software design.

Criteria:

1. The d

presented, and feasible.

2. The design is completely and appropriately implemented in the safety software.

3. The des

Approach

Review the appropriate documents, including design documents, review records,

listings. The design may be documented in a standalone document or embedded in other

docum

The software design descr

—A description of the major safety components of the software design as they relate to

the software requirements, and any interactions with n

—A t

—A description of inputs and outputs including allowable or prescribed ranges for in

and outputs.

—A description of error handling strategies and the u

—The design described in a manner suitable for translating into computer codes.

Evidence of reviews of the design and code for the appropriate grading exists. This may

overlap with the software V&V work activity.

Evidence of developer testing including any independent testing for the appropriate

grading exists.

F.5.7 SOFTWARE SAFETY

ive

ign of the safety software

software modules will perform their intended safety function in a consistent manner during

bases conditions.

1. Software systems are analyzed at the component level to ensure adequate safeguard

implemented to eliminate or mitigate the potential occurrence of a software defect that

could cause a system failure.

Safety software is designed with s

DOE G 414.1-4 APPENDIX F

6-17-05 F-15

ew hazard analysis documents to ensure that software component and interface

failures are included. This analysis may be part of a software or system level failure

iew how the identified hazards are resolved. Various methods are used for hazards

resolutions, such as eliminations, reduction of exposure, and controlling or minimizing

• ware, and optionally for Level B safety software, sample safety

odules

ion of the system, evaluate the software design for the

implementation of fault tolerant and/or self-diagnostics techniques.

F.5.8 SOFTWARE VERIFICATION AND VALIDATION

2. rmal conditions have been evaluated for mitigating unintended functions

Appro

Review

review records, desk check records, inspection repor

qualification plans and reports, and supplier qualification reports to determine whether—

Approach:

• Revi

modes and effects analysis, fault-tree analysis, event-tree analysis or other similar

analysis techniques.

• Rev

the effects of a hazard.

• Review that the hazard analysis is periodically reassessed throughout the software

life-cycle and the changes incorporated as appropriate.

For Level A safety soft

software modules for proof of design complexity evaluation and isolation of safety

functions from nonsafety functions.

• For Level A safety software, and optionally for Level B where safety software m

defects could impact the safe operat

Objective:

The V&V process and related documentation for software are defined and maintained to ensure

that (1) the software correctly performs all its intended functions; and that (2) the software does

ot perform any adverse unintended function. n

Criteria:

1. Safety software deliverables have been verified, and validated for correct operation using

reviews, inspections, assessments, observation, and testing techniques.

Relevant abno

through testing, observation, or inspection techniques.

Traceability of safety software requirements to software design and acceptance testing

has been performed.

4. New versions of the safety software are verified and validated to ensure that the safety

software meets the requirements and does not perform any unintended functions.

5. V&V activities are performed by competent staff other than those who developed the

item being verified or validated. This may overlap with the training work activity.

ach:

appropriate documents, such as SQA plans, review plans, walkthrough records, peer

ts, test plans, test cases, test reports, system

APPENDIX F DOE G 414.1-4

F-16 6-17-05

• reviews and inspections of the software requirement specifications, procurement

training materials, and user

entation have been performed by staff other than those who developed the item;

• software design was performed prior to the safety software being used in operations;

re V&V are documented and controlled;

&V methods include any one or a combination of design reviews, alternate

calculations, and tests performed during program development; and

e of standardization; (3) the similarity with previously proved

cumentation for development, factory or acceptance testing, installation, and

erations testing exists;

ults documentation demonstrates successful completion of all test cases or the

the test

— of

tandard problems with known solutions, and (5) confirmed published

opriate,

Objective:

Formal procedure for software problem reporting, and corrective action for safety software errors

and failures are established, maintained, and controlled.

• management process exists for performing V&V and management and independent

technical reviews;

documents, software design, code modules, test results,

docum

• for design V&V—

—results of the safety softwa

—V

—the extent of V&V methods chosen are a function of (1) the complexity of the

software; (2) the degre

software; and (4) the importance to safety; and

• for test V&V—

—do

—documentation includes test guidelines, test procedures, test cases including test data,

and expected results;

—res

resolution of unsuccessful test cases and proves direct traceability between

results and specified software design;

—test V&V activities and their relationship with the software life-cycle are defined;

software requirements and system requirements are satisfied by the execution

integration, system and acceptance testing;

acceptable methods for evaluating the software test case results i— nclude (1) analysis

without computer assistance, (2) other validated computer programs, (3) experiments

and test, (4) s

data and correlations;

—traceability exists from software requirements to design and testing, and if appr

to user documentation; and

—hardware and software configurations pertaining to the test V&V are specified.

F.5.9 SOFTWARE PROBLEM REPORTING AND CORRECTIVE ACTION

DOE G 414.1-4 APPENDIX F

6-17-05 F-17

umented practices and procedures for reporting, tracking, and resolving problems or

issues are defined and implemented.

are

3. Organizational responsibilities for reporting issues, approving changes, and implementing

rective actions are identified and found to be effective.

ts to both

Appro

ments and interview facility staff for the problem reporting and notification process

to determine whether—

•

the operation of the software are promptly reported to affected

•

EVAL

Objective:

Personnel are trained/qualified and capable of performing assigned work. Continuing training to

maintain job proficiency is provided.

ists for each of the following personnel

assignments:

—safety software analysis,

Criteria:

1. Doc

2. An evaluation process exists for determining if the reported problem is a safety softw

defect, error, or something else.

cor

4. For safety software defects and errors, the defect or error is correlated with the

appropriate software engineering elements, identified for potential impact, and all users

are notified.

5. For acquired safety software, procurement documents identify the requiremen

the supplier and purchaser to report problems to each other.

ach:

Review docu

a formal procedure exists for software problem reporting and corrective action

development that addresses software errors, failures, and resolutions;

problems that impact

organizations;

corrections and changes are evaluated for impact and approved prior to being

implemented;

• corrections and changes are verified for correct operation and to ensure that no side

effects were introduced;

• preventive measures and corrective actions are provided to affected organizations in a

timely manner; and

the organizations responsible for problem reporting and resolution are clearly defined.

F.5.10 TRAINING PERSONNEL IN THE DESIGN, DEVELOPMENT, USE, AND

UATION OF SAFETY SOFTWARE

personnel to

Criteria:

1. A training or indoctrination program ex

APPENDIX F DOE G 414.1-4

F-18 6-17-05

oftware development (concept to retirement),

y the training program provides for continuing education.

ould

include

underg

Unclassified Controlled Nuclear Inform

1. Title Page (Cover). The cover and title page state the name of the site, facilities assessed,

ents.

2. e report content

signature

pa logistical

nd sign

3. Ta entify all sections and subsections of

, charts, and appendices.

4. Acronyms. Include a list of acronyms used in the assessment report.

6. In uction should provide information and background regarding

ble

—s

—operations and use, and

—assessment or evaluation of safety software.

2. The training/indoctrination provides for continuing education and training to improve

their performance and proficiency.

3. Training/indoctrination is commensurate with the scope, complexity, and importance of

the tasks and the education, experience, and proficiency of the person.

Approach:

• Review training records or other documentation and conduct interviews to confirm a

training or indoctrination program exists for each of the personnel assignments listed

above.

• Verif

• Verify the training program is adequate and appropriate for the scope, complexity, and

importance of the task being performed.

F.6. REPORT FORMAT

The report is intended for cognizant facility managers and DOE line management, and sh

the sections described below. The report should conform to security requirements,

o classification review if needed, and should not contain classified information or

ation.

and dates of assessm

Signature Page. All team members, signifying their agreement as to th

and conclusions reached in the areas to which they were assigned, should sign a

ge. In the event all team member signatures cannot be obtained due to

considerations, the assessment team leader should obtain members’ concurrence a

for them.

ble of Contents. The table of contents should id

the report, illustrations

Executive Summary. The executive summary should provide an overview of the

assessment scope, any tailoring, and assessment results.

troduction. The introd

the site, facility, system, team composition, methodology, and any definitions applica

to the review.

DOE G 414.1-4 APPENDIX F

6-17-05 F-19 (and F-20)

7. .

8. As ent criteria are satisfied and describe any

co t

in activities that were not assessed and any limitations to the

10. Detailed Results. In each work activity assessed, include sufficient detail to enable a

this section

is as follows.

Is the criterion met? [Yes/No]

A, and titles of persons interviewed).

• System operability issues or concerns.

Opportunities for improvement.

11. umber, revision, and issue dates.

work papers.

Tailoring. Identify any tailoring of the criteria and guidelines provided in this CRAD

State the basis for the tailoring.

sessment Results. State whether the assessm

exceptions. Summarize opportunities for improvement and include a qualitative

nclusion regarding the ability of the system to perform its safety functions in its curren

ndition and to remain reliable over its life-cycle. Recommended actions may also be

cluded. Note any work

qualitative conclusion. A detailed discussion of results in each work activity that was

assessed should be included as a separate attachment or appendix.

Lessons Learned. Identify lessons learned that may be applied to future reviews.

knowledgeable individual to understand the results. The suggested format for

•

• How the review was conducted (include lists of documents reviewed, including

any system software documentation and Q

•

• Recommended changes to criteria and guidance.

Documents and References. Title, n

12. Assessment Data. Attach assessment records, including lines of inquiry, pertinent

assessor notes, and other relevant

13. Biographies of Team Members. Include brief biographies of all assessment team

members.

DOE G 414.1-4 APPENDIX G

6-17-05 G-1

Guide. uch as DOE Orders and the quality assurance Rule, may be

APPENDIX G. REFERENCES

The following referenced documents were used in developing the information contained in this

Some of these documents, s

obtained through the online DOE Directives, Regulations, and Standards Web site:

http://www.directives.doe.gov

. Other documents, such as the American Society of Mechanical

ers(ASME), American Society for Quality, and International Electrotechnical

ssion (IEC) standards and guidance documents may be purchased or obtained from the

ring organizations.

Engine

Commi

sponso

Repository at

5. ional Standards Institute (ANSI)/American Nuclear Society (ANS)

10.4-1987 (R1998), Guidelines for the Verification and Validation of Scientific and

Engineering Computer Programs for the Nuclear Industry, American Nuclear Society,

1998.

6. ANSI/American Society for Quality Control E4-1994, Specifications and Guidelines for

Quality Systems for Environmental Data Collection and Environmental Technology

Programs—Requirements with Guidance for Use, American Nuclear Society, 1994.

7. American Society of Mechanical Engineers (ASME), Re: Comments on the Benefits of

National Nuclear Quality Assurance Standards for NNSA and DOE Nuclear Activities

and Oversight, Letter to Linton F. Brooks, NNSA (2002).

8. ASME NQA-1-1997, Quality Assurance Requirements for Nuclear Facility Applications,

American Society of Mechanical Engineers, 1997.

9. ASME NQA-1-2000, Quality Assurance Program for Nuclear Facilities, American

Society of Mechanical Engineers, 2001.

10. ASME NQA-1A-1999, Addenda to ASME NQA-1-1997, Quality Assurance

Requirements for Nuclear Facility Applications, American Society of Mechanical

Engineers, 1999.

11. CE-1001-STD-Rev.2, Standard for Software Engineering of Safety Critical Software,

CANDU Computer Systems Engineering Centre for Excellence, Atomic Energy of

Canada, Ltd., and Ontario Power Generation, Inc., 1999.

12. Christensen, Mark J., and Richard H. Thayer, The Project Manager’s Guide to Software

Engineering’s Best Practices, Institute of Electrical and Electronics Engineers (IEEE)

Computer Society Press, 2001.

1. 10 CFR 50, Appendix B, Quality Assurance Criteria for Nuclear Power Plants and Fuel

Reprocessing Plants.

2. 10 CFR 63, Disposal of High-Level Radioactive Wastes in a Geologic

Yucca Mountain, Nevada.

10 CFR 830, Nuclear Safety Management.

4. 10 CFR 70, Domestic Licensing of Special Nuclear Material.

American Nat

APPENDIX G DOE G 414.1-4

G-2 6-17-05

CMMI for Systems Engineering, Software Engineering,

14.

16.

e Nuclear Facilities, Defense Nuclear Facilities Safety Board Technical

17. FSB

Safety

18. .61, DoD Modeling and Simulation

&S) reditation (VV&A), U.S. Department of

rtment of Defense,

20. ment, Safety and Health, Designation of

21. DOE Letter and Report, Quality Assurance for Safety-Related Software at Department of

Energy Defense Nuclear Facilities, DOE Response to TECH-25, U.S. Department of

Energy, October 2000.

22. DOE G 414.1-1A, Management Assessment and Independent Assessment Guide for Use

with 10 CFR, Part 830, Subpart A, and DOE O 414.1A, Quality Assurance; DOE P

450.4, Safety Management System Policy; and DOE P 450.5, Line ES&H Oversight

Policy, dated 5-31-01.

23. DOE G 414.1-2, Quality Assurance Management System Guide for use with

10 CFR 830.120 and DOE O 414.1, dated 6-17-99.

24. DOE G 420.1-1, Nonreactor Nuclear Safety Design Criteria and Explosives Safety

Criteria Guide for use with DOE O 420.1, Facility Safety, dated 3-28-00.

25. DOE O 414.1B, Quality Assurance, dated 4-29-04.

26. DOE O 414.1C, Quality Assurance, dated 6-17-05.

13. Capability Maturity Model Integration (CMMI) Product Team, Capability Maturity

Model Integration, Version 1.1,

Integrated Process Development, and Supplier Sourcing (CMMI-SE/SW/IPPD/SS, V1.1),

Continuous Representation, CMU/SEI-2002-TR-011, ESC-TR-2002-011,Carnegie

Mellon University (CMU) Software Engineering Institute (SEI), 2002.

CMMI Product Team, Capability Maturity Model Integration, Version 1.1, CMMI for

Software Engineering (CMMI-SW, V1.1), Staged Representation, CMU/SEI-2002-

TR-029, ESC-TR-2002-029, CMU SEI, 2002.

15. Defense Nuclear Facilities Safety Board (DNFSB) Recommendation 2002-1, Quality

Assurance for Safety-Related Software, Defense Nuclear Facilities Safety Board,

September 2002.

DNFSB/TECH-25, Quality Assurance for Safety-Related Software at Department of

Energy Defens

Report, January 2000.

DN /TECH-31 Engineering Quality into Safety Systems, Defense Nuclear Facilities

Board Technical Report, March 2001.

Department of Defense (DoD) Instruction 5000

(M Verification, Validation, and Acc

Defense, dated 5-13-03.

19. DoD MIL-STD-882D, Standard Practice for System Safety, U.S. Depa

dated 2-10-2000.

Department of Energy (DOE), Office of Environ

Initial Safety Analysis Toolbox Codes, Memorandum to Linton Brooks, Defense

Programs and Jessie Hill Roberson, Office of Environmental Management, March 28,

2003.

DOE G 414.1-4 APPENDIX G

6-17-05 G-3

27. DOE, Framework for Grading Safety Software for DOE Directive, Work Paper, U.S.

Department of Energy, dated 4-22-04.

cumented Safety Analyses, dated 7-94.

Assurance for Safety in Nuclear Power

s Q1–Q14, IAEA, 1996.

t to

35.

nic Safety-Related Systems, IEC, 1998.

38. nce Plans, IEEE, 2002.

41. IEEE,

42. 998, IEEE Standard for Software Verification and Validation—

IEEE, 1998.

44. ion Management, Section 3.3.4,

28. DOE-RW-0333P, Quality Assurance Requirements and Description, Office of Civilian

Radioactive Waste Management Program, dated 8-23-04.

29. DOE-STD-1172-2003, Safety Software Quality Assurance Functional Area Qualificatio

Standard, dated 12-03.

30. DOE-STD-3009-94, Preparation Guide for U.S. Department of Energy Nonreactor

Nuclear Facility Do

31. Herrmann, Debra S., Software Safety and Reliability: Techniques, Approaches, and

Standards of Key Industrial Sectors, IEEE Computer Society, 2000.

32. International Atomic Energy Agency (IAEA) Safety Guide (SG) Series No. NS-G-1.1

Software for Computer Based Systems Important to Safety in Nuclear Power Plants,

IAEA, 2000.

33. IAEA Safety Series No. 50-C/SG-Q, Quality

Plants and other Nuclear Installations, Code and Safety Guide

34. IAEA Technical Reports Series No. 397, Quality Assurance for Software Importan

Safety, IAEA, 2000.

IEC 61508 Parts 1–7, Functional Safety of Electrical/Electronic/Programmable

Electro

36. IEC 61511 Parts 1–3, Functional Safety—Safety Instrumented Systems for the Process

Industry Sector, IEC, 2003.

37. IEC 880, Software for Computers in the Safety Systems of Nuclear Power Stations, IEC,

1986.

IEEE Std 730-2002, Standard for Software Quality Assura

39. IEEE Std 828-1998, IEEE Standard for Software Configuration Management Plans,

IEEE, 1998.

40. IEEE Std 1008-1987(R1993), Software Unit Testing, IEEE, 1993.

IEEE Std 1012-1998, IEEE Standard for Software Verification and Validation,

1998.

IEEE Std 1012a-1

Supplement to 1012,

43. IEEE Std 1028-1997, IEEE Standard for Software Reviews, IEEE, 1997.

IEEE Std 1042-1987, IEEE Guide to Software Configurat

“Audits and Reviews,” IEEE, 1987.

APPENDIX G DOE G 414.1-4

G-4 6-17-05

46.

48.

49. are Engineering Terminology,

50. 2, IEEE Standard for Software Quality Assurance Plans, IEEE, 2002.

52. EE Standard for Software Test Documentation, IEEE, 1998.

57. IA Std 12207.1-1997, Industry Implementation of International Standard

58. 1997, Industry Implementation of International Standard

59.

-2, Configuration Management, Vital Safety Systems, dated

60. r Facilities Safety Board

ISO Quality Management

es for the Application of ISO

45. IEEE Std 1058-1998, IEEE Standard for Software Project Management Plans, IEEE

1998.

IEEE Std 1063-1987(R1993), IEEE Standard for Software User Documentation, IEEE

1993.

47. IEEE Std 1219-1993, Standard for Software Maintenance, IEEE, 1993.

IEEE Std 1228-1994, IEEE Standard for Software Safety Plans, IEEE, 1994.

IEEE Std 610.12-1990, IEEE Standard Glossary of Softw

IEEE, 1990.

IEEE Std 730-200

51. IEEE Std 7-4.3.2-2003, IEEE Standard Criteria for Digital Computers in Safety Syste

of Nuclear Power Generating Stations, IEEE, 2003.

IEEE Std 829-1998, IE

53. IEEE Std 830-1998, Recommended Practice for Software Requirements Specifications,

IEEE,1998.

54. IEEE Std 982.1-1998, IEEE Standard Dictionary Of Measures To Produce Reliable

Software, IEEE, 1998.

55. IEEE Std 982.2-1988, IEEE Guide For The Use Of IEEE Standard Dictionary Of

Measures To Produce Reliable Software, IEEE, 1988.

56. IEEE/EIA Std 12207.0-1996, Industry Implementation of International Standard

ISO/IEC 12207: 1995: Information Technology—Software Life Cycle Processes, IEEE

and the Electronic Industries Alliance (EIA), 1996.

IEEE/E

ISO/IEC 12207: 1995: Information Technology—Software Life Cycle Processes—Life

Cycle Data, IEEE and EIA, 1997.

IEEE/EIA Std 12207.2-

ISO/IEC 12207: 1995: Information Technology—Software Life Cycle Processes—

Implementation Considerations, IEEE and EIA, 1998.

IP 2000-2, Implementation Plan for Defense Nuclear Facilities Safety Board

Recommendation 2000

10-31-00.

IP 2002-1, Implementation Plan for Defense Nuclea

Recommendation 2002-1, Quality Assurance for Safety-Related Software at Departme

of Energy Defense Nuclear Facilities, dated 3-13-03.

61. International Organization for Standardization (ISO) 9000-3,

and Quality Assurance Standards—Part 3: Guidelin

DOE G 414.1-4 APPENDIX G

6-17-05 G-5 (and G-6)

64. 85, IEEE Standard for Software Engineering: Software Life

65. ddison Wesley, 1995.

are

68. er S., Software Engineering: A Practitioner’s Approach, McGraw Hill,

69.

70. chniques, Processes, and Measures for Software Safety and

71. ftware Risk Management: A Practical

9001:1994 to the Development, Supply, Installation and Maintenance of Computer

Software, ISO, 1997.

62. ISO 9001-1994, Quality Systems—Model for Quality Assurance in Design, Development

Production, Installation and Servicing, ISO, 1994.

63. ISO 9001-2000, Quality Management Systems—Requirements, ISO, 2000.

ISO/IEEE Standard 160

Cycle Processes, Risk Management, IEEE, 2004.

Leveson, Nancy, Safeware: System Safety and Computers, A

66. National Aeronautics and Space Administration (NASA) NASA-STD-2201-93, Softw

Assurance Standard, NASA, 1992.

67. NASA-STD-8719.13A, Software Safety, NASA, 1997.

Pressman, Rog

1992.

Society of Automotive Engineers (SAE), SAE JA1003, Software Reliability Program

Implementation Guide, SAE, 2004.

Sparkman, Debra, Te

Reliability, Lawrence Livermore National Laboratory, UCRL-ID 108725, 1992.

SQAS21.01.00-1999 (Reference Document), So

Guide, Department of Energy Quality Managers Software Quality Assurance

Subcommittee, February 2000.