Review past reports for outstanding issues or previous problems. Consider:

Review management’s response to issues raised during the previous regulatory examination and during internal and external audits performed since the last examination. Consider:

Interview management and review the operations information request to identify:

Describe the operational organization structure for technology operations and assess its effectiveness in supporting the business activities of the institution.

Review documentation that describes, or discuss with management, the technology systems and operations (enterprise architecture) in place to develop an understanding of how these systems support the institution’s business activities. Assess the adequacy of the documentation or management’s ability to knowledgeably discuss how technology systems support business activities.

Review operations management MIS reports. Discuss whether the frequency of monitoring or reporting is continuous (for large, complex facilities) or periodic. Assess whether the MIS adequately addresses:

Response times and throughput;

System availability and/or down time;

Number, percentage, type, and causes of job failures; and

Average and peak system utilization, trends, and capacity.

Obtain documentation of or discuss with senior management the probability of risk occurrence and the impact to IT operations. Evaluate management’s risk assessment process.

Obtain copies of, and discuss with senior management, the reports used to monitor the institution’s operations and control environment. Assess the adequacy and timeliness of the content.

Determine whether management coordinates the IT operations risk management process with other risk management processes such as those for information security, business continuity planning, and internal audit.

Review and consider the adequacy of the environmental survey(s) and inventory listing(s) or other descriptions of hardware and software. Consider the following:

Computer equipment – vendor and model number;

Network components;

Names, release dates, and version numbers of application(s), operating system(s), and utilities; and

Application processing modes:

On-line/real time;

Batch; and

Memo post.

Review systems diagrams and topologies to obtain an understanding of the physical location of and interrelationship between:

Hardware;

Network connections (internal and external);

Modem connections; and

Other connections with outside third parties.

Obtain an understanding of the mainframe, network, and telecommunications environment and how the information flows and maps to the business process.

Review and assess policies, procedures, and standards as they apply to the institution’s computer operations environment and controls.

Determine whether management has implemented and effectively utilizes operational control programs, processes, and tools such as:

Performance management and capacity planning;

User support processes;

Event/problem management.

Determine whether management has implemented appropriate daily operational controls and processes including:

Scheduling systems or activities for efficiency and completion;

Monitoring tools to detect and preempt system problems or capacity issues;

Daily processing issue resolution and appropriate escalation procedures;

Secure handling of media and distribution of output; and

Control self-assessments.

Determine whether management has implemented appropriate human resource management. Assess whether:

The organizational structure is appropriate for the institution’s business lines;

Management conducts ongoing background checks for all employees in sensitive areas;

Segregation and rotation of duties are sufficient;

Management has policies and procedures to prevent excessive employee turnover; and

There are appropriate policies and controls concerning termination of operations personnel.

Review the institution’s enterprise-wide data storage methodologies. Assess whether management has appropriately planned its data storage process, and that suitable standards and procedures are in place to guide the function.

Review the institution’s data back-up strategies. Evaluate whether management has appropriately planned its data back-up process, and whether suitable standards and procedures are in place to guide the function.

Review the institution’s inventory of data and program files (operating systems, purchased software, in-house developed software) stored on and off-site. Determine if the inventory is adequate and whether management has an appropriate process in place for updating and maintaining this inventory.

Review and determine if management has appropriate back-up procedures to ensure the timeliness of data and program file back-ups. Evaluate the timeliness of off-site rotation of back-up media.

Identify the location of the off-site storage facility and evaluate whether it is a suitable distance from the primary processing site. Assess whether appropriate physical controls are in place at the off-site facility.

Determine whether management performs periodic physical inventories of off-site back-up material.

Determine whether the process for regularly testing data and program back-up media is adequate to ensure the back-up media is readable and that restorable copies have been produced.

Review the environmental controls and monitoring capabilities of the technology operations as they apply to:

Electrical power;

Telecommunication services;

Preventive maintenance.

Assess whether controls exist to address telecommunication operations risk, including:

Alignment of telecommunication architecture and process with the strategic plan;

Monitoring of telecommunications operations such as downtime, throughput, usage, and capacity utilization; and

Assurance of adequate availability, speed, and bandwidth/capacity.

Determine whether there are adequate security controls around the telecommunications environment, including:

Controls that limit access to wiring closets, equipment, and cabling to authorized personnel;

Secured telecommunications documentation;

Appropriate telecommunication change control procedures; and

Controlled access to internal systems through authentication.

Discuss whether the telecommunications system has adequate resiliency and continuity preparedness, including:

Telecommunications system capacity;

Telecommunications provider diversity;

Telecommunications cabling route diversity, multiple paths and entry points; and

Redundant telecommunications to diverse telephone company central offices.

dentify and review the institution’s use of item processing and document imaging solutions and describe the imaging function.

Describe or obtain the system data flow and topology.

Evaluate the adequacy of imaging system controls including the following:

Physical security;

Data security;

Documentation;

Error handling;

Program change procedures;

System recoverability; and

Vital records retention.

Evaluate the adequacy of controls over the integrity of documents scanned through the system and electronic images transferred from imaging systems (accuracy and completeness, potential fraud issues).

Review and assess the controls for destruction of source documents (e.g., shredded) after being scanned through the imaging system.

Determine whether management is monitoring and enforcing compliance with regulations and other standards, including if imaging processes have been reviewed by legal counsel.

Assess to what degree imaging has been included in the business continuity planning process, and if the business units reliant upon imaging systems are involved in the BCP process.

Determine if there is segregation of duties where the imaging occurs.

Describe and assess the event/problem management program’s ability to identify, analyze, and resolve issues and events, including:

Escalation of operations disruption to declaration of a disaster; and

Collaboration with the security and information security functions in the event of a security breach or other similar incident.

Assess whether the program adequately addresses unusual or non-routine activities, such as:

Production program failures;

Production reports that do not balance;

Operational tasks performed by non-standard personnel;

Deleted, changed, modified, overwritten, or otherwise compromised files identified on logs and reports;

Database modifications or corruption; and

Forensic training and awareness.

Determine whether there is adequate help desk support for the business lines, including:

Effective issue identification;

Timely problem resolution; and

Implementation of effective preventive measures.

Assess the controls in place for processing of customer transactions, including:

Transaction initiation and data entry;

Batch processing;

Balancing;

Check in-clearing;

Review and reconcilement;

Determine the need to proceed to Tier II procedures for additional review related to any of the Tier I objectives.

From the procedures performed, including any Tier II procedures performed:

Document conclusions related to the effectiveness and controls in the operations environment; and

Determine and document to what extent, if any, you may rely upon the procedures performed by the internal and external auditors in determining the effectiveness of the operations controls.

Review your preliminary conclusions with the examiner in charge (EIC) regarding:

Noncompliance with supervisory guidance.

Discuss your findings with management and obtain proposed corrective action. Relay those findings and management’s response to the EIC.

Document your conclusions in a memo to the EIC that provides report ready comments for all relevant sections of the FFIEC report of examination.

Develop an assessment of operations sufficient to contribute to the determination of the Support and Delivery component of the Uniform Rating System for Information Technology (URSIT) rating.

Organize your work papers to ensure clear support for significant findings and conclusions.