Audit - Appendix A Examination Procedures

Booklet: Audit
Section: Appendix A: Examination Procedures

Examination objectives allow the examiner to determine the quality and effectiveness of the audit function related to IT controls. These procedures will disclose the adequacy of audit coverage and to what extent, if any, the examiner may rely upon the procedures performed by the auditors in determining the scope of the IT examination.

	Tier I objectives and procedures relate to the institution’s implementation of an effective audit function that may be relied upon to identify and manage risks.

	Tier II objectives and procedures provide additional validation as warranted by risk to verify the effectiveness of the institution’s audit function. Tier II questions correspond to the Uniform Rating System for Information Technology (URSIT) rating areas and can be used to determine where the examiner may rely upon audit work in determining the scope of the IT examination for those areas.

TIER I OBJECTIVES AND PROCEDURES

Objective 1: Determine the scope and objectives of the examination of the IT audit function and coordinate with examiners reviewing other programs.

1.	Review past reports for outstanding issues, previous problems, or high-risk areas with insufficient coverage related to IT. Consider:
		Regulatory reports of examination;
		Internal and external audit reports, including correspondence/communication between the institution and auditors;
		Regulatory, audit, and security reports from key service providers;
		Audit information and summary packages submitted to the board or its audit committee;
		Audit plans and scopes, including any external audit or internal audit outsourcing engagement letters; and
		Institution’s overall risk assessment.

2.	Review the most recent IT internal and external audit reports in order to determine:
		Management’s role in IT audit activities;
		Any significant changes in business strategy, activities, or technology that could affect the audit function;
		Any material changes in the audit program, scope, schedule, or staffing related to internal and external audit activities; and
		Any other internal or external factors that could affect the audit function.

3.	Review management’s response to issues raised since the last examination. Consider:
		Adequacy and timing of corrective action;
		Resolution of root causes rather than just specific issues; and
		Existence of any outstanding issues.

4.	Assess the quality of the IT audit function. Consider:
		Audit staff and IT qualifications, and
		IT audit policies, procedures, and processes.
Using the results from the preceding procedures and discussions with the EIC, select from the following examination procedures those necessary to meet the examination objectives. Note: examinations do not necessarily require all steps.

Objective 2: Determine the quality of the oversight and support of the IT audit function provided by the board of directors and senior management.

1.	Review board resolutions and audit charter to determine the authority and mission of the IT audit function.
2.	Review and summarize the minutes of the board or audit committee for member attendance and supervision of IT audit activities.
3.	Determine if the board reviews and approves IT policies, procedures, and processes.
4.	Determine if the board approves audit plans and schedules, reviews actual performance of plans and schedules, and approves major deviations to the plan.
5.	Determine if the content and timeliness of audit reports and issues presented to and reviewed by the board of directors or audit committee are appropriate.
6.	Determine whether the internal audit manager and the external auditor report directly to the board or to an appropriate audit committee and, if warranted, has the opportunity to escalate issues to the board both through the normal audit committee process and through the more direct communication with outside directors.

Objective 3: Determine the credentials of the board of directors or its audit committee related to their ability to oversee the IT audit function.

1.	Review credentials of board members related to abilities to provide adequate oversight. Examiners should:
		Determine if directors responsible for audit oversight have appropriate level of experience and knowledge of IT and related risks; and
		If directors are not qualified in relation to IT risks, determine if they bring in outside independent consultants to support their oversight efforts through education and training.

2.	Determine if the composition of the audit committee is appropriate considering entity type and complies with all applicable laws and regulations. Note – If the institution is a publicly traded company, this is a requirement of Sarbanes-Oxley. Additionally, this is a requirement of FDICIA for institutions with total assets greater than $500 million.

*Objective 4: Determine the qualifications of the IT audit staff and its continued development through training and continuing education.*

1.	Determine if the IT audit staff is adequate in number and is technically competent to accomplish its mission. Consider:
		IT audit personnel qualifications and compare them to the job descriptions;
		Whether staff competency is commensurate with the technology in use at the institution; and
		Trends in IT audit staffing to identify any negative trends in the adequacy of staffing.

Objective 5: Determine the level of audit independence.

1.	Determine if the reporting process for the IT audit is independent in fact and in appearance by reviewing the degree of control persons outside of the audit function have on what is reported to the board or audit committee.

2.	Review the internal audit organization structure for independence and clarity of the reporting process. Determine whether independence is compromised by:
		The internal audit manager reporting functionally to a senior management official (i.e., CFO, controller, or similar officer);
		The internal audit manager’s compensation and performance appraisal being done by someone other than the board or audit committee; or
		Auditors responsible for operating a system of internal controls or actually performing operational duties or activities.
		Note that it is recommended that the internal audit manager report directly to the audit committee functionally on audit issues and may also report to senior management for administrative matters.

Objective 6: Determine the existence of timely and formal follow-up and reporting on management’s resolution of identified IT problems or weaknesses.

1.	Determine whether management takes appropriate and timely action on IT audit findings and recommendations and whether audit or management reports the action to the board of directors or its audit committee. Also, determine if IT audit reviews or tests management’s statements regarding the resolution of findings and recommendations.
2.	Obtain a list of outstanding IT audit items and compare the list with audit reports to ascertain completeness.
3.	Determine whether management sufficiently corrects the root causes of all significant deficiencies noted in the audit reports and, if not, determine why corrective action is not sufficient.

Objective 7: Determine the adequacy of the overall audit plan in providing appropriate coverage of IT risks.

1.	Interview management and review examination information to identify changes to the institution’s risk profile that would affect the scope of the audit function. Consider:
		Institution’s risk assessment,
		Products or services delivered to either internal or external users,
		Loss or addition of key personnel, and
		Technology service providers and software vendor listings.

2.	Review the institution’s IT audit standards manual and/or IT-related sections of the institution’s general audit manual. Assess the adequacy of policies, practices, and procedures covering the format and content of reports, distribution of reports, resolution of audit findings, format and contents of work papers, and security over audit materials.

*Objective 8: Determine the adequacy of audit’s risk analysis methodology in prioritizing the allocation of audit resources and formulating the IT audit schedule.*
1.	Evaluate audit planning and scheduling criteria, including risk analysis, for selection, scope, and frequency of audits. Determine if:
		The audit universe is well defined; and
		Audit schedules and audit cycles support the entire audit universe, are reasonable, and are being met.

2.	Determine whether the institution has appropriate standards and processes for risk-based auditing and internal risk assessments that:
		Include risk profiles identifying and defining the risk and control factors to assess and the risk management and control structures for each IT product, service, or function; and
		Describe the process for assessing and documenting risk and control factors and its application in the formulation of audit plans, resource allocations, audit scopes, and audit cycle frequency

Objective 9: Determine the adequacy of the scope, frequency, accuracy, and timeliness of IT-related audit reports.

1.	Review a sample of the institution’s IT-related audit reports and work papers for specific audit ratings, completeness, and compliance with board and audit committee-approved standards.

2.	Analyze the internal auditor’s evaluation of IT controls and compare it with any evaluations done by examiners.

3.	Evaluate the scope of the auditor’s work as it relates to the institution’s size, the nature and extent of its activities, and the institution’s risk profile.

4.	Determine if the work papers disclose that specific program steps, calculations, or other evidence support the procedures and conclusions set forth in the reports.

5.	Determine through review of the audit reports and work papers if the auditors accurately identify and consistently report weaknesses and risks.

6.	Determine if audit report content is:
		Timely
		Constructive
		Accurate
		Complete

Objective 10: Determine the extent of audit’s participation in application development, acquisition, and testing, as part of the organization’s process to ensure the effectiveness of internal controls.

1.	Discuss with audit management and review audit policies related to audit participation in application development, acquisition, and testing.

2.	Review the methodology management employs to notify the IT auditor of proposed new applications, major changes to existing applications, modifications/additions to the operating system, and other changes to the data processing environment.

3.	Determine the adequacy and independence of audit in:
		Participating in the systems development life cycle;
		Reviewing major changes to applications or the operating system;
		Updating audit procedures, software, and documentation for changes in the systems or environment; and
		Recommending changes to new proposals or to existing applications and systems to address audit and control issues.

Objective 11: If the IT internal audit function, or any portion of it, is outsourced to external vendors, determine its effectiveness and whether the institution can appropriately rely on it.

1.	Obtain copies of:
		Outsourcing contracts and engagement letters,
		Outsourced internal audit reports, and
		Policies on outsourced audit.

2.	Review the outsourcing contracts/engagement letters and policies to determine whether they adequately:
		Define the expectations and responsibilities under the contract for both parties.
		Set the scope, frequency, and cost of work to be performed by the vendor.
		Set responsibilities for providing and receiving information, such as the manner and frequency of reporting to senior management and directors about the status of contract work.
		Establish the protocol for changing the terms of the service contract, especially for expansion of audit work if significant issues are found, and stipulations for default and termination of the contract.
		State that internal audit reports are the property of the institution, that the institution will be provided with any copies of the related work papers it deems necessary, and that employees authorized by the institution will have reasonable and timely access to the work papers prepared by the outsourcing vendor.
		State that any information pertaining to the institution must be kept confidential.
		Specify the locations of internal audit reports and the related work papers.
		Specify the period of time that vendors must maintain the work papers. If work papers are in electronic format, contracts often call for vendors to maintain proprietary software that allows the institution and examiners access to electronic work papers during a specified period.
		State that outsourced internal audit services provided by the vendor are subject to regulatory review and that examiners will be granted full and timely access to the internal audit reports and related work papers and other materials prepared by the outsourcing vendor.
		Prescribe a process (arbitration, mediation, or other means) for resolving problems and for determining who bears the cost of consequential damages arising from errors, omissions and negligence.
		State that outsourcing vendors will not perform management functions, make management decisions, or act or appear to act in a capacity equivalent to that of a member of institution management or an employee and, if applicable, they are subject to professional or regulatory independence guidance.

3.	Consider arranging a meeting with the IT audit vendor to discuss the vendor’s outsourcing internal audit program and determine the auditor’s qualifications.

4.	Determine whether the outsourcing arrangement maintains or improves the quality of the internal audit function and the institution’s internal controls. The examiner should:
		Review the performance and contractual criteria for the audit vendor and any internal evaluations of the audit vendor;
		Review outsourced internal audit reports and a sample of audit work papers. Determine whether they are adequate and prepared in accordance with the audit program and the outsourcing agreement;
		Determine whether work papers disclose that specific program steps, calculations, or other evidence support the procedures and conclusions set forth in the outsourced reports; and
		Determine whether the scope of the outsourced internal audit procedures is adequate.

5.	Determine whether key employees of the institution and the audit vendor clearly understand the lines of communication and how any internal control problems or other matters noted by the audit vendor during internal audits are to be addressed.

6.	Determine whether management or the audit vendor revises the scope of outsourced audit work appropriately when the institution’s environment, activities, risk exposures, or systems change significantly.

7.	Determine whether the directors ensure that the institution effectively manages any outsourced internal audit function.

8.	Determine whether the directors perform sufficient due diligence to satisfy themselves of the audit vendor’s competence and objectivity before entering the outsourcing arrangement.

9.	If the audit vendor also performs the institution’s external audit or other consulting services, determine whether the institution and the vendor have discussed, determined, and documented that applicable statutory and regulatory independence standards are being met. Note – If the institution is a publicly traded company, this is a requirement of Sarbanes-Oxley. Additionally, this is a requirement of FDICIA for institutions with total assets greater than $500 million.

10.	Determine whether an adequate contingency plan exists to reduce any lapse in audit coverage, particularly coverage of high-risk areas, in the event the outsourced audit relationship is terminated suddenly.

Objective 12: Determine the extent of external audit work related to IT controls.

1.	Review engagement letters and discuss with senior management the external auditor’s involvement in assessing IT controls.

2.	If examiners rely on external audit work to limit examination procedures, they should ensure audit work is adequate through discussions with external auditors and reviewing work papers if necessary.

Objective 13: Determine whether management effectively oversees and monitors any significant data processing services provided by technology service providers:

1.	Determine whether management directly audits the service provider’s operations and controls, employs the services of external auditors to evaluate the servicer's controls, or receives sufficiently detailed copies of audit reports from the technology service provider.

2.	Determine whether management requests applicable regulatory agency IT examination reports.

3.	Determine whether management adequately reviews all reports to ensure the audit scope was sufficient and that all deficiencies are appropriately addressed.

CONCLUSIONS

Objective 14: Discuss corrective actions and communicate findings.

1.	Determine the need to perform Tier II procedures for additional validation to support conclusions related to any of the Tier I objectives.

2.	Using results from the above objectives and/or audit’s internally assigned audit rating or audit coverage, determine the need for additional validation of specific audited areas and, if appropriate:
		Forward audit reports to examiners working on related work programs, and
		Suggest either the examiners or the institution perform additional verification procedures where warranted.

3.	Using results from the review of the IT audit function, including any necessary Tier II procedures:
		Document conclusions on the quality and effectiveness of the audit function as related to IT controls; and
		Determine and document to what extent, if any, examiners may rely upon the internal and external auditors’ findings in order to determine the scope of the IT examination.

4.	Review preliminary examination conclusions with the examiner-in-charge (EIC) regarding:
		Violations of law, rulings, and regulations;
		Significant issues warranting inclusion as matters requiring board attention or recommendations in the report of examination; and
		Potential effect of your conclusions on URSIT composite and component ratings.

5.	Discuss examination findings with management and obtain proposed corrective action for significant deficiencies.

6.	Document examination conclusions, including a proposed audit component rating, in a memorandum to the EIC that provides report-ready comments for all relevant sections of the report of examination.

7.	Document any guidance to future examiners of the IT audit area.

8.	Organize examination work papers to ensure clear support for significant findings and conclusions.

TIER II OBJECTIVES AND PROCEDURES
The Tier II examination procedures for the IT audit process provide additional verification procedures to evaluate the effectiveness of the IT audit function. These procedures are designed to assist in achieving examination objectives and scope and may be used entirely or selectively. Tier II questions correspond to URSIT rating areas and can be used to determine where the examiner may rely upon audit work in determining the scope of the IT examination for those areas. Examiners should coordinate this coverage with other examiners to avoid duplication of effort with the examination procedures found in other IT Handbook booklets.

A. MANAGEMENT

1.	Determine whether audit procedures for management adequately consider:
		The ability of management to plan for and initiate new activities or products in response to information needs and to address risks that may arise from changing business conditions;
		The ability of management to provide reports necessary for informed planning and decision making in an effective and efficient manner;
		The adequacy of, and conformance with, internal policies and controls addressing the IT operations and risks of significant business activities;
		The effectiveness of risk monitoring systems;
		The level of awareness of, and compliance with, laws and regulations;
		The level of planning for management succession;
		The ability of management to monitor the services delivered and to measure the institution’s progress toward identified goals in an effective and efficient manner;
		The adequacy of contracts and management’s ability to monitor relationships with technology service providers;
		The adequacy of strategic planning and risk management practices to identify, measure, monitor, and control risks, including management’s ability to perform self-assessments; and
		The ability of management to identify, measure, monitor, and control risks and to address emerging IT needs and solutions.

B. SYSTEMS DEVELOPMENT AND ACQUISITION

1.	Determine whether audit procedures for systems development and acquisition and related risk management adequately consider:
		The level and quality of oversight and support of systems development and acquisition activities by senior management and the board of directors;
		The adequacy of the institutional and management structures to establish accountability and responsibility for IT systems and technology initiatives;
		The volume, nature, and extent of risk exposure to the institution in the area of systems development and acquisition;
		The adequacy of the institution’s systems development methodology and programming standards;
		The quality of project management programs and practices that are followed by developers, operators, executive management/owners, independent vendors or affiliated servicers, and end-users;
		The independence of the quality assurance function and the adequacy of controls over program changes including the:
		- parity of source and object programming code, - independent review of program changes, - comprehensive review of testing results, - management’s approval before migration into production, and - timely and accurate update of documentation;
		The quality and thoroughness of system documentation;
		The integrity and security of the network, system, and application software used in the systems development process;
		The development of IT solutions that meet the needs of end-users; and
		The extent of end-user involvement in the systems development process.

C. OPERATIONS

1.	Determine whether audit procedures for operations consider:
		The adequacy of security policies, procedures, and practices in all units and at all levels of the financial institution and service providers.
		The adequacy of data controls over preparation, input, processing, and output.
		The adequacy of corporate contingency planning and business resumption for data centers, networks, service providers, and business units. Consider the adequacy of offsite data and program backup and the adequacy of business resumption testing.
		The quality of processes or programs that monitor capacity and performance.
		The adequacy of contracts and the ability to monitor relationships with service providers.
		The quality of assistance provided to users, including the ability to handle problems.
		The adequacy of operating policies, procedures, and manuals.
		The quality of physical and logical security, including the privacy of data.
		The adequacy of firewall architectures and the security of connections with public networks.

D. INFORMATION SECURITY

1.	Determine whether audit procedures for information security adequately consider the risks in information security and e-banking. Evaluate whether:
		A written and adequate data security policy is in effect covering all major operating systems, databases, and applications;
		Existing controls comply with the data security policy, best practices, or regulatory guidance;
		Data security activities are independent from systems and programming, computer operations, data input/output, and audit;
		Some authentication process, such as user names and passwords, that restricts access to systems;
		Access codes used by the authentication process are protected properly and changed with reasonable frequency;
		Transaction files are maintained for all operating and application system messages, including commands entered by users and operators at terminals, or at PCs;
		Unauthorized attempts to gain access to the operating and application systems are recorded, monitored, and responded to by independent parties;
		User manuals and help files adequately describe processing requirements and program usage;
		Controls are maintained over telecommunication(s), including remote access by users, programmers and vendors; and over firewalls and routers to control and monitor access to platforms, systems and applications;
		Access to buildings, computer rooms, and sensitive equipment is controlled adequately;
		Written procedures govern the activities of personnel responsible for maintaining the network and systems;
		The network is fully documented, including remote and public access, with documentation available only to authorized persons;
		Logical controls limit access by authorized persons only to network software, including operating systems, firewalls, and routers;
		Adequate network updating and testing procedures are in place, including configuring, controlling, and monitoring routers and firewalls;
		Adequate approvals are required before deployment of remote, Internet, or VPN access for employees, vendors, and others;
		Alternate network communications procedures are incorporated into the disaster recovery plans;
		Access to networks is restricted using appropriate authentication controls; and
		Unauthorized attempts to gain access to the networks are monitored.

2.	Determine whether audit procedures for information security adequately consider compliance with the “Interagency Guidelines Establishing Standards for Safeguarding Customer Information,” as mandated by Section 501(b) of the Gramm-Leach-Bliley Act of 1999. Consider evaluating whether management has:
		Identified and assessed risks to customer information;
		Designed and implemented a program to control risks;
		Tested key controls (at least annually);
		Trained personnel; and
		Adjusted the compliance plan on a continuing basis to account for changes in technology, the sensitivity of customer information, and internal/external threats to information security.

E. PAYMENT SYSTEMS

1.	Determine whether audit procedures for payment systems risk adequately consider the risks in wholesale electronic funds transfer (EFT). Evaluate whether:
		Adequate operating policies and procedures govern all activities, both in the wire transfer department and in the originating department, including authorization, authentication, and notification requirements;
		Formal contracts with each wire servicer exist (i.e., Federal Reserve Bank (FRB), correspondent financial institutions, and others);
		Separation of duties is sufficient to prevent any one person from initiating, verifying, and executing a transfer of funds;
		Personnel policies and practices are in effect;
		Adequate security policies protect wire transfer equipment, software, communications lines, incoming and outgoing payment orders, test keys, etc.;
		Credit policies and appropriate management approvals have been established to cover overdrafts;
		Activity reporting, monitoring, and reconcilement are conducted daily, or more frequently based upon activity;