2. System Audit (Audit Management and Initiation)

Printer-friendly versionPrinter-friendly versionPDF versionPDF version
System Revision ID ASEMS Document Version Effective From State
4701 3.2 24/06/2022 - 00:15 Extant

2.1. Overview

2.1.0.1.

This procedure describes how an audit schedule can be developed in order to organise self-audits of the Operating Centre's/Delivery Team’s Safety Management System and Environmental Management System. The activities covered in this procedure will form the basis of the system audit process so it is important that issues are considered carefully to avoid duplication of effort or gaps in the audit process later on.

2.1.0.2.

The audit schedule should describe the scope and frequency of self-audits and set out a timeframe for their completion.

2.1.0.3.

Although this and the companion procedures have been produced primarily for use by and on behalf of  Operating Centres/Delivery Teams, they may also be used by any party carrying out audits of ASEMS compliant Safety Management Systems and Environmental Management Systems. . However, it should be noted that as they have primarily been written for use by OCs/DTs, they may refer to or use terminology specific to these teams.  

Other parties may include but are not limited to:

  1. Defence Regulators;
  2. Safety Sustainable Development & Continuity;
  3. Third Parties invited by the Chief Executive Officer;
  4. Independent Safety and/ or Environment Auditors;
  5. MOD and Top Level Budget Internal Audit Functions;
  6. Personnel seconded from other teams;
  7. Front Line Commands/User;
  8. Subject Matter Expert;
  9. Environmental and Safety Consultants.

2.1.0.4.

These procedures are flexible enough to facilitate auditing all or parts of a team’s Safety Management System and Environmental Management System.

2.1.0.5.

Throughout the procedures the term ‘Audit Client’ is used to describe the group, organisation or individual commissioning an audit, as this may be distinct from the party carrying out the audit.

2.1.1. Required Outputs

2.1.1.1.

  1. Form AAP01a/F/01 - Audit Schedule
  2. Form AAP01a/F/02 - Audit Details, Team Composition and Competence Record Form
  3. Form AAP01a/F/03 – Notification of Audit Letter.

OR

Equivalent actions and documentation that QSEP is satisfied achieve the same objectives.

2.2. Procedure

2.2.1. Step 1 – Formulate an audit schedule

2.2.1.1.

NOTE:  Where the Audit Client wishes to involve the Lead Auditor in the production of the audit schedule then Step 2 – Appoint Lead Auditor should be completed before this step.

2.2.1.2.

In order to produce an audit schedule the following should be determined:

  1. What elements of the Safety Management System and Environmental Management System the audits will cover (i.e. scope);
  2. How many audits are necessary to achieve the defined objective;
  3. How often will these audits be undertaken; and
  4. When audits are to be undertaken.

2.2.1.3.

In terms of scope the schedule should apply to:

  1. Individual Operating Centre or project level Safety Management Systems and/or Environmental Management Systems; or
  2. Several project level Safety Management Systems and Environmental Management Systems; or
  3. A team level Safety Management System and/or Environmental Management System.

2.2.1.4.

The organisation and scope of the audit schedule should depend largely on how much of the Safety Management System and Environmental Management System is in place due to the project lifecycle stage at the time of audit.. A team may decide to develop separate schedules for the Safety Management System and Environmental Management System if the systems are sufficiently distinct from each other, or combine schedules where elements are shared, similar or connected.

2.2.1.5.

As the OC/DT is responsible for the Safety Management System and Environmental Management System, the overall audit schedule should cover all the existing elements regardless of whether these are the managed by the team or a contractor.

2.2.1.6.

When developing the schedule, consideration should be given to any other planned audits that may cover aspects of safety or environmental management. These may fulfil some or all of the objectives of the audit schedule and may therefore be used as alternatives to avoid duplication of effort.

2.2.2. Audit Scope

2.2.2.1.

Although it is possible to audit the whole Safety Management System or Environmental Management System at once, this should generally be avoided unless the systems are very simple.  Care should be taken as this may require significant Auditee and Auditor resources and cause significant disruption to the OC/DT.

2.2.2.2.

The audit schedule should be divided into a number of audits each of which is a manageable task. This can be done in a number of ways:

  1. By POSMS / POEMS and team’s Safety Management System/Environmental Management System requirement - This involves the auditing of the whole project(s) against each POSMS / POEMS and team’s Safety Management System/Environmental Management System requirement in turn. This approach may cross several activities and/or projects/organisations;
  2. By POSMS / POEMS procedure – This approach allows a full audit trail to be gathered;
  3. By activity, project, organisation or geographical basis – This approach provides a full audit trail only when all departments have been assessed;
  4. By safety risk or environmental impact - Full audit trails are obtained by crossing projects, organisations or activities, although the audit can be difficult to structure.

2.2.2.3.

When deciding on how to partition the audit schedule, the following issues should be considered:

  1. Purpose of the audit;
  2. Status and importance of the systems activities;
  3. Any external requests for an audit to take place. For example, from:
    1. The delegation chain Project Manager, Team Leader, Operating Centres,Chief Executiive Officer, Secretary of State (via Defence Environment Safety Board);
    2. Other Top Level Budgets
    3. Head of Departments;
    4. Defence Regulators;
    5. QSEP;
    6. Safety Sustainable Development & Continuity; and
    7. Stakeholders (Front Line Command/User Defence Infrastructure Organisation, Chief Environmental and Safety Officers etc.) through Safety Committees.
  4. Scope of the Environmental Management System and Safety Management System;
  5. Relevant domain regulatory auditing requirements;
  6. Stakeholders’ expectations;
  7. Existing team’s audit regimes including any audits planned or recently completed by other parties;
  8. Logistics;
  9. Where different parts of the same management system are best audited together;
  10. Where elements of the Safety and Environmental Management Systems are best audited together;
  11. The auditees and auditors likely to be involved;
  12. Timeframe for implementing the management system(s); and
  13. The frequency that the system element needs to be audited (i.e. try not to group elements of the management system which are best audited at a different frequency).

2.2.2.4.

Audits of Safety Management System and Environmental Management System elements should be more frequent in the following situations where:

  1. They have not been covered or only partially covered by previous audits;
  2. A high number of non-conformances have been identified;
  3. There is a high safety risk or priority environmental impact;
  4. Accidents, incidents or occurrences with safety or environmental implications have been reported;
  5. A prescriptive legal or other standard applies;
  6. There is a demonstrable level of stakeholder interest or concern.

2.2.2.5.

There may also be a need for more frequent audits in cases where:

  1. The project is approaching a critical milestone;
  2. There has been a major change in procedures, equipment system specification or use, or environmental and safety standards;
  3. There have been major personnel changes.

2.2.2.6.

The audit schedule should be designed in sections to allow meaningful and manageable auditing to take place. This enables the audit team to check that all elements are covered in the audit programme.

It is acceptable to develop and implement a prioritised audit approach based on criticality, performance or risk associated with the system(s) to be audited. Any decisions based using an applied prioritisation approach should be justifiable, documented and supportable by appropriate evidence.

2.2.3. Audit Frequency

2.2.3.1.

The next task in formulating the audit schedule should be to set a frequency for how often each audit should be completed. Audit frequency should be kept to a minimum to reduce the likelihood of ‘audit fatigue’ in the Auditee, but frequent enough to provide assurance that the management system(s) is operating effectively.

2.2.3.2.

The frequency of audits will vary from project to project but should aim to cover each element of the management system(s) at least once every 3 years. To avoid ‘over-auditing’ it is recommended that each element of the management system should be audited no more frequently than every 6 months (this excludes follow-up checks).

2.2.3.3.

The team should refer to the relevant domain regulatory requirements to establish whether it requires a shorter minimum auditing interval (higher frequency). Audit frequency may also be influenced by stakeholders’ expectations, existing team’s regimes and outputs from any Project Review and Assurance activities.

2.2.4. Documentation and Communication

2.2.4.1.

The Audit Schedule should be used to record the scope, frequency and timing of audit(s).

2.2.4.2.

For audits where the Audit Client is not the Team Leader, it is recommended that the Auditee should be contacted at this early stage to give them advance notice of the impending audit.

2.2.4.3.

Note: Where a team uses equipment system contractors* to audit the Safety Management System or Environmental Management System, then the team should undertake sample checks on the audit schedule to ensure the procedure has been followed correctly.

*The use of these parties may be helpful in cases where it is important to demonstrate the independence of the auditors from the team.

 

2.2.5. Audit Competency Guidance (formerly AAP01a/G/01)

2.2.5.1.

There are 3 main parties involved in auditing the safety and environmental management system in POEMS and

POSMS, these being:

  • Lead Auditor – The person responsible for leading and managing an audit and audit team.
  • Auditor – A person who forms part of an audit team
  • Aspirant Auditor – A person who forms part of the audit team who is undergoing training, or other development process, in order to attain auditor status.
2.2.5.2. General attributes of all auditors

2.2.5.2.1.

Personal

Auditors at all levels should be:

  • Ethical.
  • Open-minded.
  • Diplomatic.
  • Observant.
  • Perceptive.
  • Versatile.
  • Decisive.
  • Self-reliant

2.2.5.2.2.

Knowledge and skills

All staff involved in auditing POSMS and POEMS should be able to:

  • Apply audit principles, procedures and techniques.
  • Conduct audit (or designated task) within agreed time schedule.
  • Collect information through effective interviewing, listening observing and reviewing relevant information.
  • Verify the accuracy of collected information.
  • Use correct documentation to record audit activities.
  • Prepare audit reports.
  • Maintain confidentiality.
  • Understand system standards.
  • Have an awareness of relevant laws, regulations and requirements.
  • Understand relevant environmental and safety terminology.
  • Understand environmental/safety management principles.
  • Understand relevant environmental and safety management tools.
2.2.5.3. Aspirant Auditor

2.2.5.3.1.

Initially the most important areas of experience and competence are the attributes outlined in the General Attributes section above. However, in addition aspirant auditors should:

  • Have some knowledge of, and the ability to apply (under supervision) audit processes.
  • Be proficient at effectively utilising their time during audits.
  • Provide assistance to the Lead Auditor and audit team members where required.
  • Help with the preparation and production of the audit report.
  • Understand MoD Safety and Environmental management requirements.
  • Have knowledge of ASEMS, POEMS and POSMS.
2.2.5.4. Auditor

2.2.5.4.1.

An Auditor is expected to:

  • Have successfully completed an accredited auditing course (eg ISO 14001, 9001, OHSAS 18001) or have equivalent practical training and experience.
  • Have gained experience in the entire audit process by participating in a minimum of two audits, including undertaking document review and audit reporting.
  • Be proficient at effectively utilising their time during audits.
  • Provide assistance to aspirant auditors. 
  • Help with the preparation and production of the audit report.
  • Understand MoD safety and environmental management requirements.
  • Have knowledge of ASEMS, POEMS and POSMS.
2.2.5.5. Lead Auditor

2.2.5.5.1.

A Lead Auditor is expected to:

  • Have successfully completed an accredited auditing course (eg ISO 14001, 9001, OHSAS 18001) or have equivalent practical training and experience.
  • Have acted as an auditor in at least two complete audits.
  • Advise on and interpret requirements of audit processes with sufficient breadth of experience, knowledge and depth of understanding, to be able to apply audit management requirements.
  • Generate an effective auditing strategy and plan, based on the identified audit requirements.
  • Be proficient at planning and effectively utilising resources during audits.
  • Organise and direct audit team members.
  • Provide guidance and assistance to aspirant auditors.
  • Lead the audit team to reach the audit conclusions.
  • Prepare, complete and review the audit report.
  • Understand MoD safety and environmental management requirements.
  • Have knowledge of ASEMS, POEMS and POSMS and domain functional policy requirements. Do not forget that any audit must be able to inform the functional Boards that policy is being implemented effectively.

Note that whilst Lead Auditors are required to have competencies in auditing and ASEMS, it is not necessary for them to be competent with the domain of the equipment and services being audited.  The Lead Auditor can call on auditors with domain competence, or SMEs to support, or be part of, the audit team.

2.2.6. Step 2 – Appoint the Lead Auditor

2.2.6.1.

The following should be considered when appointing the Lead Auditor:

  1. Auditing competency;
  2. Knowledge of POSMS and POEMS and any relevant domain specific regulation;
  3. Equipment system and domain knowledge;
  4. Personal attributes; 
  5. Security clearance as necessary; and
  6. Independence from the activities being audited.
2.2.7. Step 3 – Define the audit objectives, scope and criteria

2.2.7.1.

The audit schedule should define the general scope of the audit; more detail on its scope, objectives and criteria should be defined by the Audit Client and Lead Auditor.

2.2.7.2.

As part of the audit, the Audit Client may also request that the Lead Auditor:

  1. Provides recommendations to address any non-conformance identified;
  2. Reviews corrective and preventive actions proposed by Auditee; and
  3. Completes follow-up checks to confirm non-conformances have been closed out.
2.2.8. Step 4 – Check the feasibility of the audit

2.2.8.1.

The Auditee should be given sufficient notice that an audit will be taking place and be made aware of the objectives, scope and criteria of the audit. This will not only remind the Auditee of the planned audit, but should also allow the feasibility of undertaking the audit as timetabled to be confirmed. A Notification of Audit Letter may be used for this purpose.

2.2.8.2.

Factors that will affect the feasibility of undertaking the audit at a particular time should include the availability of:

  1. Sufficient and appropriate information to plan the audit; and
  2. Adequate time and resources of the Auditee and Auditors.

2.2.8.3.

If it has been determined that it is not feasible to undertake the audit, an alternative solution should be agreed between the Audit Client, Lead Auditor and Auditee.

2.2.9. Step 5 – Select the audit team

2.2.9.1.

Depending on the scope, size, and timescale of the audit, the Audit Team may consist of only the Lead Auditor, or it may consist of a number of auditors. When selecting members of the Audit Team, the following should also be considered:

  1. Audit objectives, scope and criteria;
  2. Independence of the Audit Team and the entity being audited;
  3. Audit timescales;
  4. Auditor availability; 
  5. Competence of Audit Team to achieve audit objectives; and
  6. Geographical dispersion of the sites to be audited.

2.2.9.2.

It is reasonable to include Trainee Auditors within the Audit Team as a means of improving their competence, as long as the trainee auditor is not permitted to audit without appropriate direction and guidance from a competent auditor(s). On particularly large or complex audits  administrative support should be available within the Audit Team.  It should be possible to meet skills or knowledge requirements through the inclusion of an auditing expert or Subject Matter Expert to support the Audit Team.

2.2.9.3.

Further information on establishing and evaluating auditor competency can be found in Auditor Competency Interim Guidance sheet.

2.2.10. Step 6 – Contacting the Auditee

2.2.10.1.

The Lead Auditor should contact the Auditee to arrange an initial visit prior to the  audit fieldwork phase. This should take place no less than 1 month before the audit to allow the Auditee sufficient time to prepare.

2.2.10.2.

The objectives of this initial visit include:

  1. For the Auditee to understand the purpose of the audit;
  2. To enable audit methodology, limitations and timetable to be discussed;
  3. For the Auditee to meet the Lead Auditor (or team member) and for them to explain who has been appointed on the Audit Team;
  4. To establish Auditee role/contribution to the audit (e.g. to provide a guide to escort the team during the audit and provide access to areas, documentation and staff);
  5. To identify staff to be interviewed and their availability;
  6. To agree office and support arrangements for the Audit Team;
  7. For the Lead Auditor to gain an understanding of the area(s) to be audited;
  8. To identify documentation which will be required to be examined before and during the audit;
  9. To confirm confidentiality of documentation; and
  10. To facilitate the production of the audit plan.

2.2.10.3.

Where the Lead Auditor considers that an initial site visit is not appropriate or required, then planning for the audit should be made by letter/e-mail etc. Issues to consider in deciding whether a site visit is required are as follows:

  1.  Existing familiarity with the area being audited;
  2. Type, scope and depth of audit; and
  3. Travel time and/or costs (note: this is not a sufficient reason on its own).

2.2.10.4.

.The Lead Auditor may also utilise a Pre-Audit Questionnaire where they consider that this would  benefit the audit process. The time the Auditee should need to complete the questionnaire should be minimal and the questionnaire should only be used as an information gathering exercise to assist in the audit planning and document review stage, not as a replacement for work which should be completed during the audit fieldwork phase.

2.3. Responsibilities

2.3.1. Procedure Management and Procedure Completion

2.3.1.1.

The diagram below shows the steps described in the Description section of this procedure against those parties or individuals that may be responsible for their completion.

AAP01 Flow Chart

2.3.1.2.

Note that where the Lead Auditor has responsibility for completion of a task, this may be delegated to members of the Audit Team.

2.4. When

2.4.0.1.

This procedure can be applied to the Safety Management System or Environmental Management System at any time during its implementation, it is not necessary for the full system to be in place before planning and carrying out audits.

2.4.0.2.

A team will be expected to have produced an audit schedule and audited each element of its Safety Management System and Environmental Management System before Main Gate. Auditing will continue throughout the life of the project(s).

2.5. Required Inputs

2.5.0.1.

Inform,ation available from the Safety and Environmental Case(s), for example:

  • Results of previous Audits (Form AAP01d/F/01);
  • Record of Management Reviews;
  • Record of Monitoring and Measurement;
  • Environmental Management Plans (Form EMP06/F/03);
  • Safety Management Plans (outputs from SMP03 - Safety Planning);
  • Non-Conformance and Corrective Actions;
  • Register of Stakeholder Requirements (Form EMP01/F/01 and SMP01/F/02);
  • Register of Standards (Form EMP01/F/02 and EMP03/F/01 SMP01/F03);
  • List of Operational Controls (Form EMP07/F/01 and outputs from SMP08 - Risk Reduction);
  • Other POSMS and POEMS outputs.

Audit schedules produced by other parties where these cover auditing all or some of the elements of the Safety Management System and Environmental Management System.

2.6. Required Outputs

2.6.0.1.

  1. Form AAP01a/F/01 - Audit Schedule
  2. Form AAP01a/F/02 - Audit Details, Team Composition and Competence Record Form
  3. Form AAP01a/F/03 – Notification of Audit Letter.

2.6.0.2.

OR

Equivalent actions and documentation that QSEP is satisfied achieve the same objectives.

2.6.1. Records and Project Documentation

2.6.1.1.

Where relevant, the outputs from this procedure should be revised during the audit planning stage.

2.7. Further Guidance

2.7.1. General

2.7.1.1.

Although auditing Front Line Commands is out of the scope of these audit procedure, information provided by Front Line Commands in showing compliance with the Safety Management System and Environmental Management System requirements, and required equipment system safety and environmental performance (e.g. objectives and targets and operational controls) should be reviewed as part of the audit.

2.7.1.2.

If necessary, further guidance on the application of this procedure may be obtained from QSEP. 

2.8. Version Control

2.8.1. Version 2.3 to 3.0 Uplift

2.8.1.1.

Major uplift from the Acquisition System Guidance (ASG) to online version. POEMS has undergone major revision. Refer to the POEMS Transition Document for details.

2.8.2. Version 3.0 to 3.1 Uplift

2.8.2.1.

Text change replacing Project Team with Delivery Team.

2.8.3. Version 3.1. to 3.2 Uplift

2.8.3.1.

Update to Form SMP01/F/03 - Register of Safety Legislation and Other Significant Requirements to align with the form available on the DLST

More like this