Virtual Server Hosting Services (Intelligent Infrastructure): Service Level Expectations

Last updated: January 1, 2019

Related policies

• Security of Information Technology Resources (IT-12)
• Cyber Risk Mitigation Responsibilities (IT-28)

• 1 General overview
  • 1.1 Terms and definitions
• 2 Service descriptions
  • 2.1 Service scope
    • 2.1.1 Service exclusions
  • 2.2 IU Data Centers IUB and IUI
  • 2.3 Operating parameters
  • 2.4 System level
• 3 Roles and responsibilities
  • 3.1 Customer obligations
    • 3.1.1 Virtual Server System administration
    • 3.1.2 Charges (if applicable)
    • 3.1.3 Assumptions
    • 3.1.4 Hardware and infrastructure technology updates
    • 3.1.5 Backup and removal of data
  • 3.2 Service provider requirements
    • 3.2.1 Physical hardware - system administration
    • 3.2.2 Problem determination
    • 3.2.3 Backups/storage of backups
    • 3.2.4 Network services
• 4 Hours of coverage, support, response times, and escalation
  • 4.1 Hours of system administration support
  • 4.2 Service requests
  • 4.3 Service request priorities and response times
    • 4.3.1 Normal incident processing
    • 4.3.2 Major incident handling
      • 4.3.2.1 Service request priorities and response times
      • 4.3.2.2 Major incident response times
  • 4.4 Maintenance management
    • 4.4.1 Service maintenance/change management
    • 4.4.2 General exceptions to the standard maintenance window
• 5 Reporting, reviewing, and auditing
  • 5.1 Term and termination
  • 5.2 Service Level Expectation (SLE)

1 General overview

University Information Technology Services (UITS) provides Virtual Server Hosting Services to the Indiana University campuses. This Service Level Expectation (SLE) is specific to the Virtual Server Hosting Services known as the Intelligent Infrastructure (II). Unlike co-location or other physical server hosting services, II is a service where a virtual server is leased and the customer is not required to make an initial capital investment in buying physical server and storage hardware.

This is an SLE between II customers and UITS. The scope of this document includes:

• Services provided by UITS to II customers
• Levels of response time, availability, and support associated with these services
• Responsibilities of the UITS service provider and responsibilities of the customer
• Processes for requesting services and getting support

This SLE covers the period from January 1, 2019 to December 31, 2019, and will be reviewed and revised at the end of this period.

OR

This SLE shall remain valid until revised or terminated.

1.1 Terms and definitions

1. Backup solutions: Optional service available for subscription that provides cross-site backups and cross-campus failover options, which isolate you from potential disasters by securing your backup data within hardened data centers
2. Business day: Normal working day in the time zone where Indiana Data Center facilities are located (Eastern Time Zone -5GMT and participates in Daylight Saving Time)
3. Customer: The party identified as the purchasing organization to this Agreement with UITS
4. Designated contacts: Customer-named contacts, technical resources, and fiscal account resources which are established, person-specific email addresses associated with the customer support contract. It is expected that these contacts will be updated upon any personnel or responsibility change by the Customer.
5. Intelligent Infrastructure (II): Virtual Server Hosting Services, where unlike "co-location" or other physical server hosting services, a virtual server is leased and the customer is not required to make an initial investment in buying capital equipment such as servers and storage hardware. On-call support coverage for service outages is 24 hours a day, 7 days a week, 365 days a year.
6. Problem resolution: The use of reasonable commercial efforts to resolve the reported problem. These methods may include (but are not limited to) configuration changes, patches that fix an issue, replacing failed hardware, reinstalling software, etc.
7. Respond: Addressing the initial request and taking ownership of the issue
8. Response time: The amount of time elapsed between the initial contact by the customer to UITS and the returned response to the customer by UITS staff
9. Service Level Expectation (SLE): The customer Service Level Expectation (SLE) that identifies the features and defines the processes involved with the delivery by UITS of various support functions to customers, as presented by this document's content
10. Service Request (SR): A single issue opened with UITS. The SR number identifies the Service Request. The format for the unique SR number can be as follows: SAV #nnnnnnn.
11. Severity Definitions for Intelligent Infrastructure:
    1. Severity 1 (Urgent):
       1. An error with a direct security impact on the service
       2. An error isolated to the Virtual System production environment that renders the Virtual System inoperative or causes the Virtual System to fail catastrophically; for example, critical system impact, system down
       3. A reported defect in the production environment, which cannot be reasonably circumvented, in which there is an emergency condition that significantly restricts the use of the product to perform necessary business functions
       4. Inability to use the product or critical impact on operation requiring an immediate solution.
    2. Severity 2 (High):
       1. An error isolated to the Virtual System that substantially degrades the performance of the service or materially restricts business; for example, major system impact, temporary system hanging
       2. A reported defect in the Virtual System, which restricts the use of one or more features of the Virtual System to perform necessary business functions but does not completely restrict the use of the Virtual System
       3. Ability to use the Virtual System, but an important function is not available, and operations are severely impacted
    3. Severity 3 (Medium):
       1. An error isolated to the Virtual System that causes only a moderate impact on the use of the service; for example, moderate system impact, performance/operational impact
       2. A reported defect in the Virtual System that restricts the use of one or more features of the Virtual System to perform necessary business functions, while the defect can be easily circumvented
       3. An error that can cause some functional restrictions but does not have a critical or severe impact on operations
    4. Severity 4 (Low):
       1. A reported anomaly in the Virtual System environment that does not substantially restrict the use of one or more features of the Virtual System to perform necessary business functions; a minor problem and not significant to operations
       2. An anomaly that may be easily circumvented or may need to be submitted to UITS as an enhancement request
12. UITS: University Information Technology Services, which is staffed by professional support personnel providing assistance with diagnosis and resolution of defects and/or failures in II services
13. Virtual System: The method to supply the infrastructure and network capacity necessary to host your applications, while optional disk storage on UITS enterprise-class SANs (Storage Area Networks) ensures your files are highly secure and available
14. VMware: The virtualization platform is built on a business-ready architecture and uses software such as VMware vSphere to transform or "virtualize" the hardware resources of an x86-based computer including the CPU, RAM, hard disk, and network controller to create a fully functional virtual system that can run its own operating system and applications just like a "physical" server. For more about virtualization and the benefits it can provide, see VMWare's introduction to virtualization site.
15. Workaround: A change in the environment or data to avoid error without substantially impairing use of the II service

2 Service descriptions

2.1 Service scope

1. For virtual servers with IU's II Environment, the services supporting virtual systems include, but are not limited to the following components:
   1. Virtualized server infrastructure (CPU, RAM, and hard disk)
   2. VMware's High Availability functionality
   3. Data Center Operations support and monitoring
   4. Network infrastructure to provide connectivity to the virtual server
   5. Back-up services with off-site storage
2. There are two principal components to the II service package:
   1. Virtual systems supply the infrastructure for compute, disk, and network capacity necessary to host applications. Disk storage on UITS enterprise-class SANs ensures files are highly secure and available.
   2. Backup solutions provide cross-site backups and cross-campus failover options, which are isolated from potential disasters by securing backup files within hardened IU Data Centers. Virtual system rentals include a disaster recovery off-site backup of your operating system disk. Data protection services are offered as a subscription-based model.

2.1.1 Service exclusions

1. Support does not include the following items or actions:
   1. Step-by-step assistance for installation of operating system or service packs
   2. On-site services
   3. Installation or configuration of applications hosted on virtual systems
   4. Modifications of software code, security-policy configuration, audits, or security design
2. UITS shall have no obligation to support:
   1. Problems caused by customer negligence, misuse, misapplication, or use of the product beyond the control of the UITS
   2. Unsupported operating systems:
      1. II-Enterprise-supported operating systems are specifically listed on the VMware Guest Operating System guide.
      2. System owners should only install and manage operating systems that are covered by security patches; for example, while it is technically possible to install Windows 2008 in VMware, based upon the end of life of the product, it should not be used for services.
   3. Products installed, intentionally or unintentionally, that result in nefarious activities

2.2 IU Data Centers IUB and IUI

IU has two hardened data centers, one each on the Bloomington and Indianapolis campuses. The Data Centers provide a safe and secure location for IT equipment.

For more about IU's Data Centers, see Data Center options for housing departmentally owned IT equipment.

2.3 Operating parameters

Data Center Operations trained operators provide support for the Data Center 24 hours a day, 7 days a week.

For more about Data Center Operations and the services they provide, see the specific information at UITS Hardware.

2.4 System level

Storage and Virtualization (SAV) Systems Administrators provision the CPU, RAM, hard disk, and network resources for the virtual servers in VMware. The virtual server creation and configuration provides the infrastructure for storage, network, security administration, and account management of the virtual server. Ongoing support includes monitoring, performance tuning, and software patches of the physical servers hosting the virtual server.

3 Roles and responsibilities

3.1 Customer obligations

1. Customer responsibilities and/or requirements include:
   1. Staffing: All customer personnel contacting UITS for support must be fully trained on the operating system running in the virtual system.
   2. Named designated contacts:
      1. Customer-named contacts, technical resources, and fiscal account resources, which are established, person-specific email addresses associated with the customer support contract. It is expected that these contacts will be updated upon any personnel or responsibility change.
      2. Customer Active Directory Services (ADS) group to be used for Virtual System resource assignment. The ADS group contents are managed by the customer, thereby providing the most control over resource access to the customer.
      3. Customer group email distribution list to be used for notification of service-impacting events which involve that specific virtual server. The group email should be an IU address and membership should be managed by the customer, providing control to distribute the information to the desired user base.
   3. Full responsibility for system administration: System administration falls into, but is not limited to, the following areas:
      1. Initial creation of the VM through the utilization of II's self-service portal
      2. Installation and licensing of all operating system and application software
      3. VMware Tools services:
         • VMware Tools software installation is required to maintain a supported infrastructure. Installation of VMware Tools is required for each virtual server. It is strongly recommended to include VMware Tools updates as part of normal operating system patch cycles. The VMware Tools version will be evaluated as part of any debugging endeavor. In the event that the Virtual System has a VMware Tools version that is not current, upgrading VMware Tools will be the first step in solving the problem.
      4. Support and maintenance of all operating systems and application software, including the timely application of all patches and upgrades
      5. Configuration of network address through Domain Name Services (DNS) administrators at UITS
      6. Configuration of machine room firewall ports through Network administrators at UITS
      7. Security measures, particularly the establishment of appropriate authentication and authorization processes, application of operating system and application security patches, and the performance and resolution of the University Information Security Office (UISO) scans. For general security information, see Information Security Best Practices.
      8. Data management, as prescribed by university policies and state and federal laws and regulations in respect to protection of, access to, and confidentiality of institutional or personal data residing on or processed by the system
      9. Liaison or manager who will provide operations staff with support escalation and contact information for system administration functions. Contact information for billing and operational inquiries.
      10. Optional backup services are offered that provide cross-site backups and cross-campus failover options, which isolate potential disasters by securing backup data within hardened Data Centers. In the event the optional services are not utilized, data protection is solely the responsibility of the customer.
      11. Active incident response plan as outlined by University Information Security Office (UISO); see Report an IT incident .
      12. Managing system logs for operating system and application-related troubleshooting
      13. Regular scheduled auditing for abnormal events including intrusion detection
      14. If production systems are deemed critical, test VMs should be installed and maintained. In the event a vulnerability is discovered, the test VM can be utilized to test the fix quickly and deploy it with confidence in the production environment.
      15. In the event that Critical data is stored within an II VM, the client is responsible for securing data approval from the appropriate Data Stewards. The environment is managed to standards that allow for storage of critical data.
      16. Security of Information Technology Resources (IT-12) and Cyber Risk Mitigation Responsibilities (IT-28) require that university organizational units manage technology resources securely.
      17. A default layer of data protection is included as part of the storage subsystem supporting the II-E environment. All storage servicing the II-E environment leverages AES-256 encryption at rest as part of the base installation. The encryption at rest will address a significant concern related to IT-12 for data storage. Note that this addresses data at rest only. Data storage within a virtual machine still requires responsible administrative protection by the operating system administrator.
2. UISO reserves the right to audit the security of any system residing in its facilities through periodic security scans, per IT-12: "Proactively seek out and apply vendor-supplied fixes necessary to repair security vulnerabilities, within a timeframe commensurate with the level of risk (that is, within 24 hours for high-risk, with 48 hours for medium-risk, and within 72 hours for low-risk)".
3. If a system becomes compromised, UITS will immediately remove it from the network and notify the customer. The system will not be allowed back on the network until the customer has resolved the situation and UISO has certified the resolution.

3.1.1 Virtual Server System administration

1. At the physical host level, review logs, performance, system status, resource usage, and events that may result in security issues and identify any required performance tuning.
2. Maintain base hypervisor and network security. This includes hypervisor patching, firewall settings, and associated infrastructure components of the virtual hosts. If a secured base system is compromised via the application layer, the SAV System Administrator has the right to disconnect the machine from the network.

3.1.2 Charges (if applicable)

Customer billing for services will occur monthly.

3.1.3 Assumptions

1. Services are clearly documented at Intelligent Infrastructure.
2. Major II upgrades will be treated as a project outside the scope of this document.
3. Changes to services will be communicated and documented.

3.1.4 Hardware and infrastructure technology updates

1. The physical compute resources serving the II workload are hosted on high-end enterprise-class x86 hardware. The x86 hardware has an expected lifecycle replacement of approximately every 36 months. The replacement process may require the virtual system to be momentarily power cycled to complete the migration process. The migration process can be scheduled during normal customer maintenance activities at the convenience of the customer.
2. The storage supporting the II environment is hosted on high-end enterprise-class SAN. The SAN lifecycle replacement occurs between 48 and 60 months. The replacement process does require virtual systems to be interrupted.
3. Patch processing for the x86 hardware, hypervisor, and SAN occur concurrently and do not require virtual systems to be interrupted during updates.
4. UITS will provide adequate hardware for both x86 compute and storage required to support the customer workload.

3.1.5 Backup and removal of data

1. To reconstruct lost or altered customer files, data, or programs, customers must maintain a separate backup system or procedure that is not dependent on the software or hardware products under support.
2. Prior to termination of services, customers must maintain a separate backup system or procedure that is not dependent on the software or hardware products under II services.
3. Upon termination of services, the virtual system and data will be securely erased in accordance with IU IT policies and procedures. All programs and data that were served via the II offering will no longer be accessible.

3.2 Service provider requirements

SAV System Administration responsibilities and/or requirements include:

3.2.1 Physical hardware - system administration

1. At the physical hardware level (hypervisor), review logs and performance counters to obtain system status required to identify and correct potential hardware problems.
2. Apply critical patches as recommended for the virtual environment.
3. Perform system tuning as needed to the physical server environment.
4. Assign space and manage permissions and security groups.
5. Coordinate with vendors for any maintenance or support requests.
6. Capacity planning for physical resources (physical servers, SAN storage)

3.2.2 Problem determination

1. Coordinate with the vendor for any required support.
2. Determine if the problem is hardware, software, or storage by reviewing the hypervisor/backup solution event logs.
3. If and when resource contention occurs due to a server host failure or over-allocation, production systems will have priority in resource allocation over test and development systems. The virtual server clusters have been designed to avoid resource contention; however, the potential exists.

3.2.3 Backups/storage of backups

Optional backup services are offered that provide cross site backups and cross campus failover options, which isolate potential disasters by securing backup data within hardened Data Centers. The customer is responsible for ensuring the level of protection is adequate for their needs; UITS is responsible for maintaining media services to host the backup content.

For more about the levels of data protection offered, configuration of your virtual server for backups, and how to verify the level of protection you are receiving, see Data Protection Services (DPS) backups and VM disk tagging.

3.2.4 Network Services

Provide and support physical and logical network infrastructure; act as a liaison to UITS Network Engineering for problem reports and incident handling.

4 Hours of coverage, support, response times, and escalation

4.1 Hours of system administration support

The Virtual System request queue for support requests is monitored Monday-Friday 8am-5pm, with the exception of university holidays.

4.2 Service requests

1. Clients currently have the ability to self-service the creation of new Virtual Systems or update components (for example, RAM, CPU, and storage increases) for existing virtual servers. These tasks can be completed by using II's self-service portal.
2. In support of services outlined in this document, UITS will respond to service-related incidents and/or change requests submitted by the customer through Virtual System request.
   Note:

   Do not submit a service request for a Severity 1 issue via the web request form. For a Severity 1 case, contact UITS directly by telephone (812-855-9910), and request that a Severity 1 incident be opened with the SAV group related to II services. An incident number will be generated and sent to the customer via email. Provide and include any additional details that may be relevant to the case.

4.3 Service request priorities and response times

*Target response time is defined as the time between receipt of the call and the time that a Support Team member begins working on the problem. Due to the wide diversity of problems that can occur, and the methods needed to resolve them, response time is not defined as the time between the receipt of a call and problem resolution. UITS does not guarantee the resolution of a problem within the times specified.

4.3.1 Normal incident processing

1. In the event that a customer accidentally or incorrectly assigns a request priority, UITS will correct the priority by utilizing the severity definitions. Communication with the customer will occur for any priority change.
2. Service providers supporting this service will prioritize incoming service incidents as normal priority unless the service incident fits one or more of the criteria listed in the major incident handling section of this document.
3. When an IT Request ticket is opened for a client via the web interface:
   1. The Support Center will respond to the customer and process all new IT Request tickets within 8 business hours.
   2. Low (Severity 4) priority incidents will be resolved within 30 days with a status provided every five days.
   3. Medium (Severity 3) priority incidents will be resolved within five days with a daily status provided.

4.3.2 Major incident handling

1. UITS staff supporting this service will prioritize an incoming incident request as high priority if it meets any one of the following criteria:
   1. Significant number of people affected
   2. Organizational structure is a multiplier for number of people affected
   3. Percentage of total tasks that can no longer be performed by individuals
   4. Academic and Administrative Calendar deadlines
   5. Significant impact on the delivery of instruction
   6. Significant or lasting impact on student academic performance
   7. Significant risk to law, rule, or policy compliance
2. Urgent (Severity 1) priority incidents will be resolved within eight business hours with a status provided every two hours.
3. High (Severity 2) priority incidents will be resolved within one day with a status provided every six hours.
4. The infrastructure is protected and supported by vendor support 7 days a week, 24 hours per day. If incidents are linked to vendor related components, an appropriate level support case will be opened with the vendor. The customer will be updated by SAV staff with case progress.

4.3.2.1 Service request priorities and response times

*Target response time is defined as the time between receipt of the call and the time that a Support Team member begins working on the problem. Due to the wide diversity of problems that can occur and the methods needed to resolve them, response time is not defined as the time between the receipt of a call and problem resolution. UITS does not guarantee the resolution of a problem within the times specified.

4.3.2.2 Major incident response times

4.4 Maintenance management

Refer to the items below.

4.4.1 Service maintenance/change management

1. The II Virtual Server Hosting services adheres to the UITS Change Management process. Service providers for this service adhere to the UITS Maintenance Window Guidelines. Review the UITS Scheduled Maintenance windows.
2. All services and/or related components require regularly scheduled maintenance ("maintenance window") in order to meet established service levels. These activities may render systems and/or applications unavailable for normal user interaction.
3. Due to the technology available within the virtual infrastructure, a maintenance window is not reserved for II. Patches are implemented to the infrastructure in a rolling mode, which ensures Virtual Systems are available during the infrastructure maintenance. UITS will use commercially reasonable efforts to make II services available with a monthly uptime percentage of at least 99.9% of the time in a given month.
4. One exception is the virtual center service, which is not required for virtual system availability. The virtual center maintenance window is reserved weekly unless a different maintenance window is required because of risk or impact to the customer. The virtual center standard maintenance window is weekly from noon to 2pm on Wednesdays:

   Time	Sunday	Monday	Tuesday	Wednesday	Thursday	Friday	Saturday
   Begin				12pm
   End				2pm

4.4.2 General exceptions to the standard maintenance window

5 Reporting, reviewing, and auditing

IU Internal Audit performs periodic audits of the II services. This document should be reviewed a minimum of once per fiscal year. However, in lieu of a review during any period specified, the current document will remain in effect.

5.1 Term and termination

1. Term: Support shall be provided in annual terms and shall be renewable to then-current support plan when UITS is notified of customer's intent to renew the existing contract, or UITS is notified of customer's intent not to renew services.
2. Termination: Customer may terminate this service via submission of a support request. Services are billed in arrears based on actual usage; charges will be processed through month of service termination.
   1. Prior to termination of services, customers must maintain a separate backup system or procedure that is not dependent on the software or hardware products under II services.
   2. Upon termination of services, virtual system and data will be securely erased in accordance with IT policies and procedures. All programs and data that were served via the II offering will no longer be accessible.

5.2 Service Level Expectation (SLE)

SLE Update: This agreement and related UITS plan offering details are operational in nature and may be modified any time by UITS. UITS will communicate in advance proposed changes to customer. The customer may terminate the customer relationship without penalty if all parties cannot abide by the revisions. This agreement supersedes any previous service level expectation.