Team PSD Design Review Policy and Procedure
Code Freeze: An event, usually after the completion of testing, where no more changes are allowed to the application code. The Code Freeze normally signifies the code in the TEST-Slow instance is ready for release.
Cross-functional: This is a requirement, design feature or other element that requires input across workgroups.
Design Review: This is a formal meeting where design documentation is presented and approved for further development. There are typically two design reviews (Wireframes review and Prototype Review) and one testing review. A design review can result in a design freeze or a code freeze to maintain the scope and schedule of product delivery.
Design Freeze: An event, usually after the acceptance of the Wire Frames design document and any prototypes where no more design features are allowed.
In-progress Review (IPR): This is a formal progress meeting to discuss design activities against the delivery schedule. The IPR also notes any challenges a team may be facing in the delivery of the product.
Instance: Also known as a “Project” in Epicenter, the instance is an individual, fully provisioned Modeling to Learn Sim UI.
Milestone: A milestone is a grouping of tasks constructive to completion of a more major task by a given date. A milestone may be functional or cross-functional.
Product: A product is a deliverable or maintained output of a workgroup. For example, a See Guide is an example of a product from the Facilitate Workgroup.
Product Group: A product group is a combination of individual products that share a dependency. For example, a change on a screen in the Sim UI may drive a dependency in a .gif file contained in a See Guide.
Project: Also known as an “Instance” in Epicenter, the project is an individual, fully provisioned Modeling to Learn Sim UI.
Prototypes: A prototype is a click-able rendition of a user interface, that help to characterize how the user will move through a given application or feature. Prototypes can be provided in Adobe XD or in html depending on needs and circumstances.
Quality Code: A quality code provides a standardized coding method for quality defects encountered during development or use of a given MTL product.
Release: A release is the integrated release of an individual or several combined products. For example, a Sim UI release may occur by itself, or in conjunction with the release of new or updated training materials. The same can happen vice versa.
Master Feature Card: The Master Feature Card is a tracking tool that tracks a customer requirement through design to testing. It defines a customer’s requirement, design features that support the requirement and test criteria that demonstrate the design criteria meets the customer requirement. The Master Feature Card will be hosted on GitHub and flow through the feature_tracker kanban.
Testing: The establishment of testing criteria in reference to a customer requirement and presentation of such a test to a user-group for validation.
User-customer Requirement: A needed capability or a solution to a problem that must be met or possessed by the system and/or product grouping to satisfy the customer need.
Validation: The process of evaluating a product or product group during and at the end of the development process to determine whether the product or product group satisfies specified requirements.
Version: A version is a stated and uniform number that indicates the alignment of the latest state-of-being among all product groupings. Whether a new development or revision of a product or a product group will result in a revision-level change is determined during Definition of user requirements process.
Wire Frames: Wire Frames communicate the first interpretive step of the translation of requirements into design features. A wire frame will include (as is appropriate) an algorithm that shows the screen flow and user interactions, sketches of screens, dialogue, pop ups and buttons.
This design process will rely on existing standing workgroup and support group leads meetings. However, at the request of any workgroup lead, representative or the PI, an ad-hoc design review can be organized at any time, to review any design. To the greatest extent possible, feedback during prototype and testing cycles will use distributed techniques to gather input. For example, if a new feature is in prototype, a link will be shared to the wider community (i.e., QIICS) with a defined review period. The matrix below describes the types of meetings that support the process and their objectives. Whenever possible, meetings requiring a decision will be scheduled at the Milestone Review meeting – or – will provide at least 10 days’ notice.
| Meeting Title | Frequency | Purpose | Objectives |
|---|---|---|---|
| Milestone Review | First Monday of each month | Review workgroup individual and cross-functional milestones and discuss the current month’s priorities. | - A prioritized list of milestones to be pursued during the month. - Identification of design review, prototype review, test review or other decision meetings that will be needed during the month. Results in: Approved Monthly Schedule. |
| Design Review | As needed | Review wire frame and RTVM as well as any related product design documentation and discuss changes and refinements. | - Review wire frame documents - Review RTVM - Adjust and refine understandings as reflected in requirements and wire frame documents as necessary. Results in: Approval for prototype development. |
| Prototype Review | As needed | Review prototype comments and any related product design documentation and discuss changes and refinements. | - Review prototype and prototype feedback. - Adjust and refine understandings as reflected in requirements, wire frame documents and prototype as necessary. Results in: Design freeze and approval for development. |
| Test Review | As needed | Review test comments and any related product design documentation and discuss changes and refinements. | - Review prototype and prototype feedback. - Adjust and refine understandings as reflected in requirements, wire frame documents and prototype as necessary. Results in: Code freeze and approval for production release. |
| Validation and Release | As needed | Review all product or product groups that are ready for release to ensure they are complete. | - Review product or product groups for quality and completeness. - Discuss any ramifications to the Community of Practice. - Discuss release date and any related supporting announcements or advertisements. Results in: Go Live Date. |
| Workgroup Leads In-progress Review (IPR) | 2nd Thursday of each month | To review workgroup individual and cross-functional tasks and task needs. | - To review status of milestone related tasks. - Discuss needs between workgroups as needed. - Identify any delays or other schedule related adjustments. |
The process shown in Figure 1 – New Product Design and Development Process is a schematic of how we gather essential requirements, create technical specifications, create a good design to answer the requirements and then build the product in a short time. The process seeks to promote concurrent updates and development in other workgroup areas, remove obstacles from product development and reduce time from the identification of new requirements to implementing the new idea into production.
Figure 1 – New Product Design and Development Process.
When a new requirement is identified in an issue it will flow through the issue_tracker triage section. There it will be reviewed by Workgroup Leads and they will determine its disposition in accordance with the Team PSD Policy and Procedure Standing Operating Procedure (SOP), Kanban Management System shown here Team PSD SOP. If an issue is not identified as a new feature, it shall be corrected as is appropriate using the Break-Fix cycle described later in the document.
If a new feature requirement is identified in the triage process, the issue will be assigned to the Work Breakdown column of the feature_tracker. New feature requirements normally exceed 8 hours of development time; however, this is only a guideline as some “bugs” sometimes exceed that amount of time. Workgroup Leads will identify any dependencies and these dependencies will be identified in the Feature Requirements Card. A release milestone with an estimated delivery date shall be created and all supporting development tasks shall be tracked underneath this milestone.
If dependencies or other concurrent workgroup efforts are identified during work breakdown, these shall be tracked under their own issue number. Furthermore, workgroup leads may, at their own discretion, provide issue placeholders for requirements that may emerge later in the development process. These issues shall be tracked under the release milestone mentioned in the paragraph above.
New requirements will be documented using the Feature Template on GitHub and will be referred to as the Master Card for the Feature. The Master Card will contain the requirement, design features of all products that may be associated with the requirements, and the testing requirements that will answer whether the design intent has been achieved. The Master Card should be available at all design review meetings in order to ensure it is up to date with respect to the customer’s voice.
- Number the requirement: Requirements are organized as a parent (requirement or use case description) with one child (functional requirement). A requirement tracking number will be constructed as shown below.
- Issue # First Level Second Level
- 999 1 1
- When combined:
- 999.1.1
- Identify the new requirements: Provide descriptive statements that are unique, linked to other requirements in the set, consistent with respect to the intent of the request, bounded by the scope of the requirement, and granular enough to enable design activities. Requirements will be modifiable throughout the design and development process to promote clarity, definition and remove redundancy.
- Organize requirements: Identify requirements that are derived from others. Ensure they are organized within the appropriate level and do not overlap with other requirements.
- Identify and describe functional requirements: Functional requirements are specific capabilities that the product must show or perform to satisfy the requirement. The functional requirement should describe, to the greatest extent possible, any conditions or constraints that may affect the function’s ability to meet the requirement.
- Priority: In some cases, it may be necessary to prioritize design activities. In these cases, the following rubric will be used to identify the priority of the design element.
- A – Develop this now
- B – This is important. If A is done, work on this
- C – It would be nice to work on this, but not critical.
- Test Case Document: Each functional requirement will be user-tested to ensure the design team achieves the design intent. A test case will be created that will encompass one or several functional requirements. The test case document will be controlled by the issue number and will cross-reference the requirement number it addresses. The Test Case Document will contain the following (please note this function may be accomplished with a user interface in the future):
- A title that reflects the given requirement(s) being tested.
- The Requirement Numbers that are being tested.
- The testers who have been identified to execute the test script.
- A record of test completion with feedback.
- Test Status: There are three testing statuses, complete, in-process or delayed. These will be indicated in the RTVM during the testing phases of a design.
- Defect Reporting: Major defects are reported in The Master Card and will be used for future design process quality improvement initiatives. Defect reporting will incorporate a coding system as shown in the matrix below. A defect may have one or more codes.
| Code | Issue Description |
|---|---|
| Sim UI Related Codes | |
| 001UI | Button not working as expected |
| 002UI | Navigation not working as expected |
| 003UI | Page not loading as expected |
| 004UI | Improper navigation |
| 005UI | Poor graphics resolution |
| 006UI | Adaptive screen resolution not working properly |
| 007UI | Incorrect spelling/grammar |
| 008UI | Incorrect graphic |
| 009UI | Incorrect color |
| 010UI | 508 non-compliance incorrect table header label |
| 011UI | 508 non-compliance incorrect table sub header label |
| 012UI | 508 non-compliance color contrast not sufficient |
| 013UI | 508 non-compliance tab order incorrect |
| Model Related Codes | |
| 001M | Incorrect model values |
| 002M | Incorrect variable mapping |
| Training Related Codes | |
| 001T | Training materials not consistent with Sim UI |
| 002T | Incorrect spelling/grammar |
| 003T | Incorrect graphic |
| 004T | Incorrect “I” information |
| 005T | Incorrect color |
| Not otherwise categorized | |
| 0001Z | Other |
Versioning is a method of providing product alignment and control. Major version increments will be handled per release (i.e. 1.0 to 2.0) and minor version increments can be released when ready (i.e. 1.1 to 1.2). Releases need to be tracked carefully so we know which version of our tools each wave of our trial's learners received and to ensure that we have fully documented and trained teammates before releasing something into the field. Careful decisions will need to be made regarding whether a change constitutes a major or minor release and should be discussed during Work Group leads meetings.
Wire Frames are a document that is provided at the first design review meeting (see Figure 2 – Wire Frames Example). The Wire Frame author shall create a new issue in GitHub, share the completed Wire Frame document with the wider community and allow for 3-5 days of review. Any resulting feedback will be maintained in the associated issue thread and incorporated into the Wire Frame document. Workgroup leads and the Principal Investigator will review feedback at the design review meeting for the Wire Frame. With or without the recommendation of the Workgroup Leads, the Principal Investigator may require a prototype for further design exploration. However, prototypes are not mandatory for the development process. If a prototype is not commissioned, the PI may issue a design freeze and development activities may commence. All Wire Frames documents will contain: - An algorithm that outlines the workflow, how the user will interact with the system, and how the system will respond. Each step will be aligned, wherever possible with a corresponding functional requirement for easy cross-reference. - Visual element mock-ups of screens, pop ups, tiles, buttons and related visuals that will appear as indicated in the algorithm. Mock-ups will have a high degree of fidelity with the graphical outputs of the test version of the application.
Figure 2 – Wire Frames Example
Prototypes are user-interaction capable design concepts. After a Wire Frames document is accepted, a prototype may be commissioned to provide an additional verification that the design intent will satisfy requirements. A prototype may be created in Adobe XD or in HTML. A prototype is a desirable but not mandatory step in the development process. The prototype author shall create a new issue in GitHub, provide a link to share the prototype with the wider community and allow for 3-5 days of review. Any resulting feedback will be maintained in the associated issue thread and incorporated into the Wire Frame document. Workgroup leads and the Principal Investigator will review feedback at the design review meeting for the Prototype. If the prototype is successful, the PI may issue a design freeze.
Once the design freeze is completed, the author shall communicate the completed designs to dependent workgroup leads and the Sim UI development team. The Sim UI development team shall provide an estimate of when the modifications will be available in TEST-Slow instance.
When the design has frozen, a Test Case Document will be created. The Test Case Document will have a title and cross reference the functional requirement and the supporting user-task that will demonstrate the achievement of the design intent. A Test Case Document may provide test guidance for one or more functional requirements. A Test Case Document is named by referencing the requirement numbers contained within the scope of the test. See figure 3 for an example Test Case Document. Feedback gathered during the testing process will be used to make corrections to the version of the Sim UI in the TEST-Slow instance. The scale of changes is assumed to be small; however, if a change is identified that will exceed the estimated hours for development, then a design review will be held to discuss the scope of the changes needed. Notwithstanding any major revisions, the results of testing and incorporated code changes will result in a code freeze.
Figure 3 – Example Test Case Document
Before release into production, workgroup leads should meet to determine if all requirements and associated dependencies are satisfied. These requirements will be documented in a checklist inside a separate GitHub issue associated with a release milestone. For example, if a requirement created a feature that required associated documentation such as training guides, or tutorials or news flash announcements, then the Leads should ensure these are correct and ready for release. They shall check these items off the checklist to indicate the requirement has been satisfied.
The Break-Fix Cycle is a fast-track design and development cycle designed to fix bugs and other previously developed features that are not functioning properly. The Break-Fix cycle shall track progress of issues using the “Bug” label and the issue_tracker. Testing shall be limited to the Sim UI team and Headquarters at their discretion. When an issue is identified in the issue_tracker the Sim UI Workgroup Lead shall ensure the issue is clearly understood and shall affix a quality code. The Sim UI Workgroup Lead may follow up with the reporting agency to ensure the problem is well understood and repeatable. The Sim UI Workgroup Lead shall provide guidance to the Development Team and oversee production of the fix. Upon completion of fix, the Sim UI Lead shall conduct testing to ensure the fix worked. If the error cannot be understood or replicated, the issue shall be assigned a quality code and deposited in the quality column of the issue_tracker. The quality column shall be reviewed from time-to-time to determine if patterns regarding possible bugs or faults can be identified.