Quality Assurance

This Quality Assurance Surveillance Plan (QASP) has been developed as the principal basis for assessing overall performance of the technical support for the Seaport E Service effort. This document will be used by the Government & Contractor  to evaluate the Contractor’s performance to assure they are timely, effective, and are delivering the services and products specified in the contract.

This QASP provides the methodology by which Imagine Media Group LLC (IMG Networks) performance will be monitored and to establish performance benchmarks that ensure a quantifiable basis for measuring effectiveness. The plan is designed to limit surveillance to the minimum necessary to verify that the Contractor is satisfactorily performing services directly related to the performance requirements of the Performance Work Statement (PWS).  The performance requirements are those designated with corresponding performance measures and performance standards.  Surveillance will be accomplished in accordance with the Performance Requirements Summary (PRS).

SURVEILLANCE

The Contracting Officer’s Representative (COR) will be identified in the PWS.  COR authority will be limited to administering specific technical aspects of the contract.  The PWS will identify specific guidelines and clearly detail the limits of the COR’s authority.  The COR will:

a.       Monitor Contractor performance in accordance with this QASP.

b.       Coordinate priorities of support, resources, and associated schedules.

SURVEILLANCE METHODS

Surveillance of IMG  performance is the method used by the Government to determine whether the contractor is effectively and efficiently complying with all terms and conditions of the Seaport E contract N00178-11-R-4000.  The COR may use the surveillance methods listed below in the administration of this QASP:

a.       Inspection:  A method carried out by visual examination of a product (e.g., meeting minutes, data from testing, plans, procedures, reports, or documentation) or process.

b.       Analysis:    A method that is carried out by detailed examination to assess the application of the techniques used in order to determine if they are appropriate and effective.

c.        Demonstration:  A method carried out by operation of a process from start to finish or the operation of the system, or part of the system, that relies on observable functional operation.

d.       Customer Feedback:  A method carried out by soliciting customer/end user satisfaction of technical services performed by the contractor.

PERFORMANCE REQUIREMENTS SUMMARY

The performance requirements requiring Government surveillance, the performance standards, and the surveillance methods are set forth in the PRS, Attachment (1).  The performance requirements correspond to  the PWS.  The performance measures to be used are technical accuracy, clarity, completeness, timeliness, and effectiveness.  The performance standards define what is considered acceptable performance and are the benchmarks against which performance will be measured.  The Acceptable Quality Level (AQL) is the maximum acceptable variation from the performance standard.  The surveillance method is how performance will be checked.  The COR will use the PRS to evaluate the IMG’s performance to assure they are timely, effective, and are delivering the services and products specified in the contract.

PERFORMANCE REVIEWS AND EVALUATIONS

COR Responsibility:  The COR will conduct monthly performance reviews and evaluations with the IMG based upon the information obtained through the surveillance methods identified above pertaining to the performance outlined in the PRS. The following criteria will be used for performance evaluations:

EXCELLENT:

Performance significantly exceeds contractual requirements to the Government’s benefit; above the minimal acceptable level.

 

SATISFACTORY:

Performance meets contractual requirements; minimal acceptable level.

 

UNSATISFACTORY:

Performance does not meet contractual requirements; below minimal acceptable level.

IMG Responsibility:  The failure of the IMG to meet the AQL set in the PRS may result in a reduction in contract payment.  Reviews and evaluations will form the basis for determining compliance with performance requirements, measures, and standards.  Performance reviews will be used to identify the degree of success or failure by IMG.  Failure to perform due to IMG delay or failure to comply with the mutually developed project schedule will be considered a failure in performance threatening the execution of the program and will result in a basis for Government consideration.

Performance Requirement Summary

Performance Requirement

PWS

Para

Performance Measure

Performance

Standard

Acceptable

Quality Level (AQL)

Surveillance

Method

Monthly Status Report

 

 

Technical Accuracy,  Clarity, and Completeness

 

The report accurately reflects:

•   All services provided and all deliverables completed during the reporting period and amount charged to the contract.

•   All planned activities/tasks for the upcoming reporting period and estimated cost of events to the government.

•   All problem areas and actions planned or recommended solutions for alleviating the problem.

 

Identifies 98% of the planned activities/tasks.

Inspection

 

 

Timeliness

The report shall be delivered no later than the 10th of each month following the reporting period.

Meets delivery schedule 100% of the time (excluding government caused delays).

 

Inspection

 

Effectiveness

Problems avoided or reduced as a result of implementing the planned or recommended solutions.

Performance occurs with no required rework at least 98% of the time.

Analysis,

Demonstration,

and

Customer Feedback

 

Implementation

 

 

 

Technical Accuracy,  Clarity, and Completeness

 

Provide engineering, network, technical, and analytical expertise required to:

  • Install, upgrade, test, and maintain all network equipment and computer software.
  • Implement network security and ensure compliance with the documents listed in PWS .
  • Troubleshoot server hardware, client machines, traffic injection devices, cryptographic end items, network simulators, and software.
  • Perform backup, archiving, and disaster recovery, as required.
  • Operate and maintain the  infrastructure.
  • Operate and maintain WAN.
  • Develop test procedures and Tactics, Techniques, and Procedures (TTPs).
  • Perform market surveys and trade-off studies of emerging networks and transport technologies and document findings in a Technical Report.
  • Assist with traffic data collection and analysis in deployed environments and on-site field visits to other units.

 

Test procedures adhere to the established format and content requirements of PWS .

 

Technical report includes summary of the results from market survey and trade-off analysis, include trade-offs between time, cost, and performance based on the parameters of the  recommended best value.

 

The final version of products produced adhere to the established format, are complete (i.e., all constituent parts are present and each part is addressed in sufficient detail based on its intended use), and incorporate Government review comments.

 

Troubleshoot, diagnose, and identify technically correct solution for problems or issues with  hardware, software, equipment, or tools 95% of the time.

 

Maintain all  systems in a mission capable status and security controls properly implemented/ configured 95% of the time.

 

Successfully implement network and system changes 100% of the time as necessitated by IA security notifications.

 

Successfully configure, operate, and maintain the 98% of the time.

 

Data collection and traffic analysis is relevant and technically accurate 98% of the time.

 

Inspection,

Analysis,

and

Demonstration

Timeliness

Performs all tasks and produces documentation per the established schedule.

 

Modifications or redlines made to the test procedures during test execution are due 7 calendar days following completion of the test.

 

The final document is due 7 calendar days after receipt of Government comments, when applicable.

 

Meets schedule at least 98% of the time (excluding Government caused delays).

 

 

Inspection

 

Effectiveness

Performance of tasks and products produced directly contribute to the successful establishment of the PWS.

 

Level of detail for products produced is based on the intended use and audience, and demonstrates a firm understanding of network design and implementation.

 

Performance occurs with no required rework at least 98% of the time.

Analysis,

Demonstration,

and

Customer Feedback

 

Deployment

 

 

 

 

Technical Accuracy,  Clarity, and Completeness

 

Provide engineering, network, technical, and analytical expertise required to:

  • Support traffic analyzer/injection device(s).
  • Assist  deployed support team with collection of raw, aggregated traffic data in order to compose unit-specific traffic profiles.
  • Conduct statistical analysis of traffic profiles for performance improvements and develop recommendations for improving network efficiency and throughput using Performance Enhancing Proxies.
  • Develop script traffic profiles for distribution .   Incorporate traffic profiles into a virtual architecture for future testing events.
  • Develop Semi-Annual Traffic Analysis Report that includes the content requirements of PWS .
  • Design, develop, and execute tests.   Develop and maintain test plans, test cases/scenarios, test procedures/ scripts, and test data collection and analysis processes.  Perform data reductions and analysis, and document the findings and results in a test report.
  • Prepare and maintain documentation that define and document the PWS.  Implement document version controls and a process for document staffing, resolution of comments, and change management.
  • Develop training materials .

 

Technical report includes summary of the technical issue, results obtained for the analytic effort, conclusions, and recommendations.

 

Test plans adhere to the format and content requirements of the Government provided template, and include test case/scenario description.

 

Test procedures adhere to the established format and content requirements of PWS .

 

Test reports adhere to the format and content requirements of the Government provided template, and PWS .

 

The final version of products produced adhere to the established format, are complete (i.e., all constituent parts are present and each part is addressed in sufficient detail based on its intended use), and incorporate Government review comments.

 

Perform data collection and traffic analysis 100% of the time when deployed and during on-site visits.

 

Data collection and traffic analysis is relevant and technically accurate 98% of the time.

 

Training materials includes 99% of the steps need to set-up, configure, install, and maintain the .

 

Test plan identifies 95% of the required resources.

 

Test procedures and scripts include 98% of the required inputs, operator steps, and expected outputs.

 

Troubleshoot, diagnose, and identify technically correct solution for problems or issues with hardware, software, equipment, or tools 95% of the time.

 

Maintain all  systems in a mission capable status and security controls properly implemented/ configured 95% of the time.

 

Successfully implement network and system changes 100% of the time as necessitated by IA security notifications.

 

Products produced adhere to the established formats 99% of the time.

 

Inspection,

Analysis,

and

Demonstration

Timeliness

Performs all tasks and produces documentation per the established schedule.

 

Modifications or redlines made to the test procedures during test execution are due 7 calendar days following completion of the test.

 

The final document is due 7 calendar days after receipt of Government comments, when applicable.

 

Meets schedule at least 98% of the time (excluding Government caused delays).

 

 

Inspection

 

Effectiveness

Performance of tasks and products produced directly contribute to the successful development of unit-specific traffic profiles.

 

Level of detail for products produced is based on the intended use and audience, and demonstrates a firm understanding of network design and implementation, and traffic analyzers/ injection devices.

 

Performance occurs with no required rework at least 98% of the time.

Analysis,

Demonstration,

and

Customer Feedback

 

Sustained Operations

 

Technical Accuracy,  Clarity, and Completeness

 

Provide engineering, network, technical, and analytical expertise required to:

  • Support capacity planning and the development of long-term strategic goals.
  • Conduct research on client/server hardware, software, and protocols in support procurement and development efforts.
  • Install, configure, and test emerging systems, software, and network protocols as they relate to improving efficiency.  Ensure technology is sufficiently up-to-date to avoid obsolescence and the associated sustainment problems, and accommodate future growth.
  • Evaluate acceleration devices for the purpose of integration into various PORs.
  • Determine the effectiveness and efficiency of existing systems; develop and implement strategies for improving and further leveraging these systems.
  • Define and implement strategies for integrating disparate operating systems environments.
  • Determine required network components to ensure data access, as well as data consistency and integrity.
  • Coordinate and perform in-depth tests, including end-user reviews, for modified and new systems.  Develop and maintain test plans, test cases/scenarios, test procedures/ scripts, and test data collection and analysis processes.  Perform data reductions and analysis, and document the findings/results in a test report.
  • Communicate application problems and issues to key stakeholders, including management, and end users.
  • Performa and test system backups and restores to ensure system recovery from error or outage.
  • Anticipate, mitigate, identify, troubleshoot, and resolve hardware and software problems on all tested systems.
  • Monitor, test, and tune system performance; preserve and provide system log files as needed.
  • Use the collected traffic profiles for investigating emerging WAN & LAN networking technologies that provide a greater benefit the current to the OSI Layer-3 and Layer-4 solutions.
  • Provide test, analysis, and profile injection support for devices.
  • Travel and assist with semi-annual collection of traffic data for analysis and follow-on test integration.

 

Technical reports include:

  • Summary of the technical issue, results obtained for the analytic effort, conclusions, and recommendations.
  • Summary of the results from market survey and trade-off analysis, include trade-offs between time, cost, and performance based on the parameters of the  recommended best value.

 

Test plans adhere to the format and content requirements of the Government provided template, and include test case/scenario description.

 

Test procedures adhere to the established format and content requirements of PWS paragraph 4.6.

 

Test reports adhere to the format and content requirements of the  provided template, and .

 

The final version of products produced adhere to the established format, are complete (i.e., all constituent parts are present and each part is addressed in sufficient detail based on its intended use), and incorporate  review comments.

 

Perform data collection and traffic analysis 100% of the time when deployed and during on-site visits.

 

Data collection and traffic analysis is relevant and technically accurate 98% of the time.

 

Successfully perform system backups and restores 99% of the time with no loss of data within the established timeframe.

 

Test plan identifies 95% of the required resources.

 

Test procedures and scripts include 98% of the required inputs, operator steps, and expected outputs.

 

Troubleshoot, diagnose, and identify technically correct solution for problems or issues with hardware, software, equipment, or tools 95% of the time.

 

Maintain all  systems in a mission capable status and security controls properly implemented/ configured 95% of the time.

 

Successfully implement network and system changes 100% of the time as necessitated by IA security notifications.

 

Products produced adhere to the established formats 99% of the time.

 

Inspection,

Analysis,

and

Demonstration

Timeliness

Performs all tasks and produces documentation per the established schedule.

 

Modifications or redlines made to the test procedures during test execution are due 7 calendar days following completion of the test.

 

The final document is due 7 calendar days after receipt of Government comments, when applicable.

 

Meets schedule at least 98% of the time (excluding Government caused delays).

 

 

Inspection

 

Effectiveness

Performance of tasks and products produced directly contribute to the successful operation and maintenance , including the continual update of traffic analysis profiles, test and evaluation of acceleration devices, and evaluation of emerging protocols.

 

Level of detail for products produced is based on the intended use and audience, and demonstrates a firm understanding of network design and implementation, traffic analyzers/ injection devices, and network integration and emerging technologies.

 

Performance occurs with no required rework at least 98% of the time.

Analysis,

Demonstration,

and

Customer Feedback

 

Reviews

 

Technical Accuracy,  Clarity, and Completeness

 

Review, analyze, and provide written comments on  system documentation to determine technical currency and ensure technical content satisfies its intended use.

 

Results of each review are documented in a Technical Report that:

  • Accurately describes technical discrepancies, errors, or omissions found in the document, including a reference to the appropriate paragraph/topic, and substantiations and recommendations
  • Propose changes/solutions that are feasible and technically accurate.
  • Present findings in a manner that is understandable to the intended audience.
  • Adhere to the established format.

 

The final version of the technical report (document review) adheres to the established format, is complete (i.e., all constituent parts are present and each part is addressed in sufficient detail based on its intended use), and incorporates Government review comments, when applicable.

 

Recommendations and proposed changes are relevant and technically accurate 98% of the time.

 

Technical Report adheres to the established format 99% of the time.

 

 

Inspection,

Demonstration,

and Analysis

 

 

 

Timeliness

Tasks performed and technical reports (document reviews) produced per the established schedule

 

A final report is due seven (7) calendar days after receipt of Government comments, when applicable.

 

Meets schedule at least 98% of the time (excluding Government caused delays).

 

Inspection

Effectiveness

Results of document reviews contribute to the early identification and resolution of problem areas.

Performance occurs with no required re-performance/rework at least 98% of the time.

Analysis,

Demonstration,

and

Customer Feedback

 

Meeting Support

 

Technical Accuracy,  Clarity, and Completeness

 

Participates in Technical Interchange Meetings, video teleconferences, planning sessions, and provide technical input regarding the collection and analysis of traffic data and traffic profiles, and make recommendations for network improvement.

 

Document meeting minutes in a Technical Report after each meeting that:

·  Accurately describe the participants, subjects/information, and tasking covered during the meeting,

  • Accurately describe the decisions, conclusions, direction, and agreements reached by the participants during the meeting.
  • Action items and due dates.
  • Recommendations to resolve technical issues and areas requiring further analysis.

 

The final version of the technical report (meeting minutes) adheres to the established format, is complete (i.e., all constituent parts are present and each part is addressed in sufficient detail based on its intended use), and incorporates Government review comments, when applicable.

 

Technical input provided during meetings is relevant and technically accurate 98% of the time.

 

Minutes include 95% of the participants, information covered, decisions, direction, recommendations or assigned action officers, and dates.

 

Technical Report adheres to the established format and is technically accurate 98% of the time.

 

 

 

 

 

 

 

 

 

 

Inspection,

Demonstration,

and Analysis

Timeliness

Participates in all meetings and produces technical reports (meeting minutes) per the established schedule.

 

A final report is due seven (7) calendar days after receipt of Government comments, when applicable.

 

Meets schedule at least 98% of the time (excluding Government caused delays).

 

Inspection

Effectiveness

Participation in meetings directly contributes to the early identification and resolution of problem areas.

 

Performance occurs with no required rework at least 98% of the time.

Analysis,

Demonstration,

and

Customer Feedback