Test programmes and methodologies (P&M) are organisational and methodological documents that apply to the testing of defence products, their components, software, and support systems.
Their provisions and applicable requirements are set out in the 1 November 2023 national standard DSTU V 15.211:2023 “Lifecycle Management System for Arms and Military Equipment. Test Programmes and Methodologies. Main Provisions”.
What are these organisational and methodological documents?
Test Programme: The programme sets out the object and purpose of the trials, the types, sequence and scope of experiments, as well as the procedure, conditions, location and timing. It also defines the resources required, reporting arrangements, and responsibilities for ensuring and conducting the tests.
Test Methodology: The methodology describes the method, conditions and means of testing, rules for selecting and/or producing samples, and the operational algorithms used to determine one or more interrelated characteristics of the object. It specifies how data should be presented, how accuracy and probability of results are assessed, and includes requirements for safety and environmental protection.
Points to note:
- Test programmes must contain lists of specific checks or evaluations to be carried out, with references to the relevant methodologies.
- P&M must also set out the conditions that determine readiness for testing, procedures for completing individual stages, and criteria for moving on to subsequent stages.
- P&M must not include provisions that allow adjustment or calibration during testing if such actions are not provided for in the instruction manual. Nor should they permit shortcuts that could undermine the reliability of results.
Test Programme: Mandatory sections & required information
- Object of Testing: Full designation of the prototype, index and drawing reference; number of items to be tested and the applicable selection procedure; set of test items; list of components and assemblies.
- Purpose of Testing: Clear objectives and tasks to be achieved.
- General Provisions: List of governing documents underpinning the tests; location and duration; participants; acceptance of the prototype by the customer’s representative; record of earlier tests confirming the prototype’s maturity; list of documents submitted for testing, updated in line with previous results, and characterising the maturity of the prototype.
- Scope of Testing: Stages and checks to be carried out; quantitative and qualitative characteristics to be assessed; sequence and regimes of testing; requirements for prototype operation during trials; requirements for software testing; post test activities, applicable requirements, scope, and procedures.
- Conditions and Procedures: Testing conditions, site characteristics, season, ambient temperature, gradients, pressure and humidity, permissible deviations; criteria for starting and completing stages, plus any limitations imposed by the testing environment; requirements for maintenance during testing and its frequency; arrangements to guarantee safe and incident free trials; interaction between participants; procedure for engaging experts to assess possible damage likely to be caused to the prototype during testing; requirements and admission procedures for test personnel, if necessary.
- Safety Measures: Core requirements for ensuring safety during testing, in line with design documentation, safety regulations, and technical standards.
- Logistical Support: Specific types of support, with functions and tasks allocated among participants and deadlines for delivery. A separate subsection covers metrological support.
- Safeguarding Classified Information: Measures to maintain secrecy; technical protection of information; and countering foreign technical intelligence.
- Reporting: List of reporting documents to be prepared during and after testing, with details of who drafts, endorses, and approves them, and deadlines. Distribution lists must be specified. Reporting documents include a test certificate and a test report, a certificate of post-testing prototype condition, source test data, write-off certificates in respect of material assets, etc. Procedures, location and timelines for the storage of source test data must also be set out.
Test Methodology: Mandatory sections & required information
- Object of Testing: Designation, index and composition of the test object, plus functional features relevant to applying the methodology.
- Purpose of Testing: The ultimate aim of verifying the characteristic, as expressed in the title of the methodology.
- General Provisions: Definition of the characteristic being verified (if not already standardised in normative terminology); the test method and justification for its choice; explanatory information relating to the test object.
- Characteristics and Indicators for Evaluation, and Calculation Relationships: List of indicators quantitatively expressing the characteristic; calculation relationships and formulas used to derive indicators to be evaluated.
- Conditions and Procedures: Testing conditions; duration, frequency, cycle of testing, and sequence of external influences to be reproduced; qualification requirements for support personnel; occupational safety, fire safety and environmental protection requirements; functional features of test objects and auxiliary means, and their interaction; volume of information to be recorded and methods of recording; forms and procedures for recording statistical data (including detailed expanded data logs); control methods applied to the prototype; sequence of operations during tests and checks, with control points, measurement methods and frequency, and measuring instruments used.
- Safety Measures: Core requirements for ensuring safety during testing, in line with design documentation, safety regulations, and technical standards.
- Processing, Analysis and Evaluation of Test Results: Use of statistical data accumulated before testing (if available); volume of information processed; statistical methods applied; processing methods and their place in the workflow; requirements for processed information; accuracy requirements; procedure and sequence of analysis, including express analysis; scope of input data needed for evaluation; method of comparing results with the requirements set out in the Test Programme; criteria for considering the item as having passed the test; criteria for sufficiency of testing.
- Logistical Support: Technical assets with names and codes; equipment required; list of measuring and recording instruments with names, codes and quantities by type; necessary design and technical documentation; simulation and modelling tools with names, codes and quantities; list and quantities of necessary materials, including consumables; vehicles involved; other types of logistical support; procedure for preparing and using resources during testing. A separate subsection covers metrological support.
- Safeguarding Classified Information: Measures to maintain secrecy; technical protection of information; and countering foreign technical intelligence.
- Reporting: Requirements for the scope of information to be recorded in the test protocol (or in material supporting the test certificate) in respect of the relevant section contained in the Test Programme.