ESMF >

MILESTONE A: Software Engineering Plan Completed

EARTH SYSTEM MODELING FRAMEWORK / CAN 00-OES-01

1.Milestone delivery

A1. Establish open development environment, include source code and document repositories.

An ESMF website has been created with links to project news and meetings, public documents, related publications and presentations, and other topics.

ESMF is registered as a software project in the SourceForge development environment under a Gnu public license. The ESMF team is currently using the SourceForge CVS repository to collaboratively develop prototype code and documents, including the ESMF Software Developer's Guide and ESMF General Requirements Document. A hands-on training session on how to navigate the SourceForge and ESMF websites and use development tools was held at NCAR in Boulder, CO, on 20 February 2002, for the project development team.

A2. Deliver draft ESMF Software Developer's Guide and ESMF General Requirements Document via the Web.

Click here for the draft ESMF Software Developer's Guide in html form. We anticipate that this document will evolve rapidly over the next few months, as we begin designing and developing code.

The ESMF General Requirements Document is the first section of an extensive, detailed draft ESMF Requirements Document that also includes requirements for specific aspects of the ESMF software, such as regridding.

A3. Select Executive Board.

The following are the members of the ESMF Executive Board:

Tim Killeen, Chair

Arlindo da Silva

John Marshall

Stephen Lord

Max Suarez

Ants Leetmaa

Quentin Stout

ex officio members: Chris Hill, Cecelia DeLuca, V. Balaji

A4. Comments on requirements solicited at community meeting.

A community meeting was held on 30 May, 2002 at the Grand Hyatt Hotel in Washington, D.C. A draft version of the ESMF Requirements Document was presented and distributed to meeting participants in hardcopy form. Comments on the document were solicited from the community, and may be submitted via an on-line form available under the "Development" link on the ESMF website. The ESMF Requirements Document is also available under this link, see A2. The comment period ends on 28 June 2002, and a revised version of the document will be made publicly available during July 2002.

It is estimated that more than 80 people participated in the community meeting, more than 60 of whom were not members of the ESMF collaboration.

A5. Definition of all documents to be delivered.

Section 3 of the ESMF Software Developer's Guide contains a listing and description of the documents that will be delivered over the course of the ESMF project. See A2 for how to obtain the ESMF Software Developer's Guide.

2. Reporting Requirements

Titles of the agreements and agreement numbers.

A High-Performance Software Framework and Interoperable Applications for the Rapid Advancement of Earth System Science. Part I: Core Earth System Modeling Framework Development (Cooperative Agreement NCC5-623, hereafter referred to as Part I )
A High-Performance Software Framework and Interoperable Applications for the Rapid Advancement of Earth System Science. Part II: Modeling Applications for the Earth System Modeling Framework (Cooperative Agreement NCC5-625, hereafter referred to as Part II )
A High-Performance Software Framework and Interoperable Applications for the Rapid Advancement of Earth System Science. Part III: Data Assimilation Applications for the Earth System Modeling Framework (GSFC Task Agreement GSFC-CT-3, hereafter referred to as Part III)

Milestone text and due date.

Basic development infrastructure completed. Establish open development environment, include source code and document repositories. Deliver draft ESMF Software Developer's Guide and ESMF General Requirements Document via the Web. Select Executive Board. Comments on requirements solicited at community meeting. Definition of all documents to be delivered.

Contact information for the lead software engineer and team members.

This information is covered in Section 2.1, "Communication", of the Draft Developers Guide submitted in partial fulfillment of this milestone. The full project (Part I, Part II and Part III) has appointed a technical lead team that oversees all three parts. The team has three members:

V. Balaji (NOAA/GFDL) vb@gfdl.gov
Cecelia DeLuca (NSF/NCAR) cdeluca@ucar.edu
Chris Hill (MIT) cnh@plume.mit.edu

The technical lead team has day-to-day responsibility for managing the software engineering process over all three projects. All project-related communication should be sent to a single electronic point of contact, esmf@ucar.edu. This will ensure that correspondance is seen by all the technical leads, is recorded and can be handled appropriately.

A written description of the application being developed.

Section 5, "Scope", of the ESMF Requirements document submitted as part of the fulfillment of this joint milestone describes the scope of the entire project, Parts I, II and II. It provides a written description of the software being developed under the project and of the pre-existing applications that are being deployed as part of the project.

The deployment work that will be carried out under Parts II and III constitutes facets of the "suite of applications demonstrating how the ESMF software is used in practice" that is described in the ESMF Requirements document Section 5. Software development effort associated with the "interface specification for coupling components" and the "reference implementation" is also described in the ESMF Requirements document Section 5. The implementation work for these latter two items pertains primarily to the Part I proposal, but the design and detailed architecture component of the development cuts across all three parts of the project (Part I, Part II and Part III). Section 5 also outlines a range of demonstration experiments that will be deployed under the Part II and III agreements. The publicly available Part I, Part II and Part III milestone table shows the final set of milestones in joint form. It lists the exact application configurations that will be deployed on the framework, under Part II and Part III, known as the Part II and III JMC codes. As indicated in Section 5 these applications are pre-existing and are being "deployed" on the framework, as opposed to being developed, through the set of agreements that this milestone is part of. The agreements in no way be construed to imply that these pre-existing applications are being developed as part of the agreements. The development activity being carried out under Part I, Part II and Part III is limited to design, implementation and validation of the ESMF framework and ancillary development required to support deployment. The text of the joint milestone document shows the exact configurations (of the pre-existing applications) that will be used to demonstrate deployment (see the table of JMC codes and the milestone I and J items) and interoperability.

Preliminary requirements definition for the application.

The ESMF Requirements document General Requirements section contains the preliminary requirements that have been jointly developed by the Part I, Part II and Part III teams. The General Requirements include specification of the requirements that the software being developed under the three agreements must meet in order to support the Part II and Part III pre-existing application deployment efforts.
Consistent with
1. the joint milestone definition in milestone A,
2. the agreed process identified at the project negotiations session
3. the nature of the Part I, Part II and Part III efforts
the General Requirements section constitutes a full and complete set of preliminary requirements for the software that is being developed under Part I, Part II and Part III projects.

Detailed requirements sections in the ESMF Requirements document contain further requirements derived partially from the deployment experiments to be carried out under the Part II and Part III agreements. These sections unambiguously show that the General Requirements are the preliminary requirements for the entire Part I, Part II and Part III effort. We refer to these sections here as they may provide useful background material for milestone A review. However, the General Requirements, as amended through multiple discussions with the NASA review team, are the sole set of preliminary requirements definition for all the software development being undertaken and for which funding has been budgeted under the Part I, Part II and Part III agreements. There is no sense in which they should be construed as requirements for only one part of the project or that they should be taken to imply Milestone A fulfillment for only one part of the project.

Strategy for phased approach to software development for the full lifecycle of the model, linked to the requirements.

The strategy for a phased approach to software development is discussed in the Draft ESMF Developers Guide submitted in partial fulfillment of this joint milestone (see for example Section 1 - Software Development Cycle in the Core Team Guide part (II) of the document) and is partially reproduced in the General Requirements document ( see section 8 - Organization and conventions of detailed requirements chapters ). As explained in the Developers Guide the strategy covers the full lifecycle of the project and allows for continuity beyond agreement completion. As stated in the aforementioned documents, each requirement in the General Requirements section of the ESMF Requirements document (and also the fourteen accompanying Detailed Requirements sections which we assume here to be treated solely as background material) includes information on the phase of the project during which requirements will be included into the ESMF system. Requirements that fall outside the timeline of the project are also included, to allow for anticipating developments over the full lifecycle of all of the software that is being jointly developed under the Part I, Part II and Part III agreements.

Design elements that enable ease-of-maintenance and robust integration of experimental modules.

Figure 1 and Section 5 in the ESMF General Requirements document describe the basic modular architecture within which framework features and deployment experiments will de developed under this agreement. As described in section 4, Configuration Management, of the Deployment Team Guide part (III) of the Draft ESMF Developers Guide that has been submitted in partial fulfillment of this joint milestone, applications used in the deployment experiments all use strict versioning systems and the deployment examples will be based on specifically documented versions of those codes. The Core Framework ease-of-maintenance and robust integration strategy is discussed in sections 1-9 of the Core Team Guide part (II) of the Draft ESMF Developers Guide. As discussed in the Developers Guide a processes for validating and tracking problems associated with the deployment examples and the core framework will be developed that ensures high-quality. Similarly, again as elaborated in the Developers Guide and also in the joint project Part I, Part II and Part III Risk Assessment document, an automated regression testing process will be developed that ensures robust integration of modules as the set of deployment experiments grows and as the feature set of the framework software being developed grows.

Plans for open source, software reuse, portability, and interoperability with other community efforts.

As described in section 4, Configuration Management, of the Deployment Team Guide part (III) of the Draft ESMF Developers Guide the deployment example applications will be made available through the web, are based on existing pools of software and will use ESMF to run efficiently on multiple platforms. Section 1.2, Communication with the Community, of the Project-Wide part of the Draft ESMF Developers Guide describes the liaisons between the ESMF project and the broader community. We are also activity engaging other community projects.

Plans for managing the software configuration and controlling multiple versions of the software.

Plans for software configuration management of the software development that has been funded under this agreement are described in the section 4, Configuration Management, of the Core Team Guide part (II) of the Draft ESMF Developers Guide. The process of handling pre-existing application codes, the development of which is independent of this project and is not funded and will not be managed directly by this project, is described in section 4, Configuration Management in the Deployment Team Guide part (III) of the Draft ESMF Developers Guide.

Plans for ensuring that accepted (and documented) software practices and processes are adopted and followed (quality assurance).

Plans for ensuring that accepted (and documented) software practices and processes are adopted and followed (quality assurance) for the software development (requirements definition, systems analysis, architecture specification, detailed design and subsequent implementation, validation and maintenance) that is funded under the Part I, Part II and Part III agreements are described in sections 1-9 of the Core Team Guide part (II) of the Draft ESMF Developers Guide that has been submitted in partial fulfillment of this joint milestone.

The plans for ensuring that accepted (and documented) software practices and processes are adopted and followed (quality assurance) for deployment demonstrations using pre-existing applications the development, documentation and quality assurance processes of which are not in any way part of this agreement and are not in any way funded through the budget of this agreement are described in sections 1-8 of the Deployment Team Guide part (III) of the Draft ESMF Developers Guide.

Plans for community engagement beyond delivery and receipt of comments.

Section 1.2, "Communication with the Community", in the the Project-Wide Guide part (I) of the Draft ESMF Developer's Guide describes a variety of mechanisms by which community feedback will be solicited and subsequently incorporated.

Plans to collaborate with the CT Evaluation Team's efforts to instrument development codes.

Section 1.3, "Communication with the NASA Computational Technologies Team", in the Project-Wide Guide part (I) of the Draft ESMF Developers Guide discusses the manner in which both the core framework code and the deployment demonstration applications will be made available to the CT Evaluation Teams.

Evaluation/audit process.

As described in section 7 of the Core Team Guide part (II) of the Draft ESMF Developers Guide and section 7 of the Deployment Team Guide part (III) of the Draft ESMF Developers Guide internal quality assurance processes will be implemented that are independent of the development and deployment efforts. The processes will ensure that code developed as part of the project meets project defined standards and requirements. Pre-existing application code that is being re-used as part of the deployment demonstration efforts will also be subject to a quality process that is independent of the deployment effort. This quality process will be defined to ensure that standards appropriate for the deployment codes are enforced. The process will include steps to ensure that deployment examples build on all test platforms (with any exclusions for specific codes documented) and are publicly available with a reasonably automated and robust install mechanism. The quality process will also include steps to ensure that the pre-existing application deployment demonstrations document the manner in which the framework has been adopted by the application.

Validation plan.

Section 6 in the Core Team Guide part (II) of the Draft ESMF Developers Guide describes the validation process for the software being developed under these agreements.
Section 6 in the Deployment Team Guide part (III) of the Draft ESMF Developers Guide requires that a validation process be identified for deployment codes by milestone H (Aug 2002), following initial analysis.

Risk assessment and mitigation plan.

A Risk Assessment document (Download PDF) has been submitted in partial fulfillment of milestone A. It includes all outstanding edits that accommodate concerns of the NASA evaluation team, as jointly agreed by the NASA evaluation team and the ESMF Technical Lead team through a series of e-mail and telephone conference review discussions. As with the ESMF Developers Guide and the General Requirements document, the Risk Assessment refers to risks and mitigation strategies that apply to the Part I, Part II and Part III efforts in their entirety. It is not intended and should not be construed as applying to only one Part of the project. It contains descriptions of both technological risks and organizational risks as well as details on the actions that are being taken to mitigate the risks at present and actions that have been identified as future possible courses of remediation to be followed as needed.

3. Outstanding Commitments

Upon review of the items delivered above, the NASA CT team requested a set of modifications and additions to be completed prior to Milestone acceptance. Revised versions of the draft ESMF Software Developer's Guide and ESMF General Requirements Document were completed as per reviewer comments and the links above have been updated. The draft Software Developers Guide now contains a number of dated placeholders for adding material to subsequent versions of the document. Most of these placeholders correspond to conventions and procedures that are more appropriately developed at a later time in the project.

In addition to the revisions described above, the following tasks will be completed by the specified dates:

addition of an administrative / management page and a related projects page to the ESMF website by June 21, 2002 (completed)
community input on requirements catalogued and viewable on the ESMF website by July 31, 2002 (completed)
ESMF website to be under version control on the SourceForge website by August 31, 2002