Configuring and Managing Storage
Configuring and Managing Storage
Storage Concepts
If you are an ESX/ESXi administrator planning to set up ESX/ESXi hosts to work with SANs, you must have a working knowledge of SAN concepts. You can find information about SANs in print and on the Internet. Because this industry changes constantly, check these resources frequently.
If you are new to SAN technology, familiarize yourself with the basic terminology.
A storage area network (SAN) is a specialized high-speed network that connects computer systems, or host servers, to high performance storage subsystems. The SAN components include host bus adapters (HBAs) in the host servers, switches that help route storage traffic, cables, storage processors (SPs), and storage disk arrays.
A SAN topology with at least one switch present on the network forms a SAN fabric.
To transfer traffic from host servers to shared storage, the SAN uses Fibre Channel (FC) protocol that packages SCSI commands into Fibre Channel frames.
To restrict server access to storage arrays not allocated to that server, the SAN uses zoning. Typically, zones are created for each group of servers that access a shared group of storage devices and LUNs. Zones define which HBAs can connect to which SPs. Devices outside a zone are not visible to the devices inside the zone.
Zoning is similar to LUN masking, which is commonly used for permission management. LUN masking is a process that makes a LUN available to some hosts and unavailable to other hosts. Usually, LUN masking is performed at the SP or server level.
Storage Virtualization
Computer systems are not the only candidates for virtualizing; storage can be virtualized, too. This section describes IBM SAN Volume Controller, which provides a virtual pool of storage consisting of SAN-attached physical storage devices.
Example : IBM SAN Volume Controller
The SAN Volume Controller (SVC) is a hardware device that brings storage devices in a SAN together in a virtual pool. This makes your storage appear as one logical device to manage. To the connected computers, SVC offers virtual disks as ordinary SCSI devices. On the SAN side, SVC integrates various storage subsystems, even multivendor, and takes care of the correct block mapping between the SAN devices and the virtual disks for the computers
The above Storage virtualization: virtual view and physical view
The following points make the SVC an attractive tool for an energyefficient storage strategy:
Data migration from older to newer, more efficient systems can happen transparently.
Tiered storage enables you to use media with a smaller energy footprint while the SVC cache improves its performance.
Consolidation of the system's individual storage devices to virtual storage has the same effect—increasing storage utilization—as is shown for server virtualization.
Storage virtualization requires more effort than server virtualization, often requiring us to rethink the existing storage landscape. During consolidation, large amounts of data must be moved from the old systems to the consolidated storage system. This can become a long task that requires detailed planning. However, when done, the effect can be enormous because now storage can be assigned to systems in the most flexible way.
Virtual Tapes
Tapes are the cheapest medium on which to store data. They offer the largest storage volume at the lowest cost, which is the reason that they are the optimal backup medium. Tapes have a long latency compared to disks. This is not always a benefit. Currently, data centers are faced with a time limitation for backing up and probably restoring their data because the time frames for backups shrink, while the amount of data to back up expands. For this reason, many sites prefer large disk-based backup systems instead of tapes.
Tape virtualization might be a solution to this problem. A virtual tape server behaves just like a tape library, but a very fast one. This is made possible with internal disk arrays and a migration strategy to export to and import from real tape libraries.
Storage Overview
DAS,FC SAN, iSCSI, FCoE, NAS
Direct-attached storage (DAS) is digital storage directly attached to the computer accessing it, as opposed to storage accessed over a computer network. Examples of DAS include hard drives, optical disc drives, and storage on external drives.
FC SAN:
A fiber channel storage area network (FC SAN) is a system that enables multiple servers to access network storage devices. A storage area network enables high-performance data transmission between multiple storage devices and servers.
FC technology is essential for SAN implementation and establishes connectivity according to requirements.
Techopedia explains Fiber Channel Storage Area Network (FC SAN)
In SAN terminology, fabric is the hardware that connects PCs and servers to storage devices.
The following are basic FC SAN components:
iSCSI : which stands for Internet Small Computer System Interface, works on top of the Transport Control Protocol (TCP) and allows the SCSI command to be sent end-to-end over local-area networks (LANs), wide-area networks (WANs) or the Internet. IBM developed iSCSI as a proof of concept in 1998, and presented the first draft of the iSCSI standard to the Internet Engineering Task Force (IETF) in 2000. The protocol was ratified in 2003.
Serial Attached SCSI (SAS) is a point-to-point serial protocol that moves data to and from computer storage devices such as hard drives and tape drives. SAS replaces the older Parallel SCSI (Small Computer System Interface, pronounced "scuzzy") bus technology that first appeared in the mid-1980s.
Encapsulation of Fibre Channel Frame in an Ethernet Packet.
With FCoE technology, servers, instead of having multiple discrete I/O adaptors for LAN and SAN traffic, will have a smaller number of converged network adapters (CNAs) that support both LAN and Fibre Channel SAN traffic
NAS : A Network Attached Storage (NAS) device is a storage device connected to a network that allows storage and retrieval of data from a centralised location for authorised network users and heterogeneous clients. NAS devices are flexible and scale-out, meaning that as you need additional storage, you can add on to what you have..
NAS for Small Businesses
Data is a critical asset for companies today. Without access to their data, companies may not be able to provide their customers with the level of service desired. Poor customer service, loss of sales or team collaboration problems are all examples of what can happen when information is not available to teams within a company.
Each of these issues contribute to lack of efficiency and potentially, loss of income if customers cannot wait for a data outage to be corrected.
Additionally, when it comes to data storage, small businesses find themselves faced with other issues, such as needing their storage solution to be:
Types of Datastore, VMFS, NFS
Data Store : A data store is a repository of a set of data objects. These objects are modelled using classes defined in a database schema. A data store is a general concept that includes not just repositories like databases, but also simpler store types such as flat files etc
VMFS & NFS:
How Virtual Machines Access Storage
Virtual machines use virtual disks for their operating system, application software, and other data files. A virtual disk is stored as a VMDK file on a datastore. The virtual disk hides the physical storage layer from the virtual machine’s operating system. Regardless of the type of storage device that your host uses, the virtual disk always appears to the virtual machine as a local SCSI device. As a result, you can run operating systems that are not certified for specific storage equipment, such as SAN, in the virtual machine.
When a virtual machine communicates with its virtual disk stored on a datastore, it issues SCSI commands. Because datastores can exist on different types of physical storage, these commands are encapsulated into other forms, depending on the protocol that the ESX/ESXi host uses to connect to the physical storage device.
To the applications and guest operating systems running on each virtual machine, the storage subsystem appears as a virtual SCSI controller connected to one or more virtual SCSI disks as shown in the top half of Storage Architecture. These controllers are the only types of SCSI controllers that a virtual machine can see and access, and include the objects that extend
VirtualSCSIController:
ParaVirtual SCSI Controller
Virtual BusLogic Controller
Virtual LsiLogic Controller
Virtual LsiLogic SAS Controller
How precisely a virtual machine accesses storage depends on the setup of the host. Storage API Architecture gives an overview of the different possibilities.
Storage API Architecture:
Networked Storage that ESXi Supports
vSphere Features Supported by Storage
Storage Concepts
If you are an ESX/ESXi administrator planning to set up ESX/ESXi hosts to work with SANs, you must have a working knowledge of SAN concepts. You can find information about SANs in print and on the Internet. Because this industry changes constantly, check these resources frequently.
If you are new to SAN technology, familiarize yourself with the basic terminology.
A storage area network (SAN) is a specialized high-speed network that connects computer systems, or host servers, to high performance storage subsystems. The SAN components include host bus adapters (HBAs) in the host servers, switches that help route storage traffic, cables, storage processors (SPs), and storage disk arrays.
A SAN topology with at least one switch present on the network forms a SAN fabric.
To transfer traffic from host servers to shared storage, the SAN uses Fibre Channel (FC) protocol that packages SCSI commands into Fibre Channel frames.
To restrict server access to storage arrays not allocated to that server, the SAN uses zoning. Typically, zones are created for each group of servers that access a shared group of storage devices and LUNs. Zones define which HBAs can connect to which SPs. Devices outside a zone are not visible to the devices inside the zone.
Zoning is similar to LUN masking, which is commonly used for permission management. LUN masking is a process that makes a LUN available to some hosts and unavailable to other hosts. Usually, LUN masking is performed at the SP or server level.
Storage Virtualization
Computer systems are not the only candidates for virtualizing; storage can be virtualized, too. This section describes IBM SAN Volume Controller, which provides a virtual pool of storage consisting of SAN-attached physical storage devices.
Example : IBM SAN Volume Controller
The SAN Volume Controller (SVC) is a hardware device that brings storage devices in a SAN together in a virtual pool. This makes your storage appear as one logical device to manage. To the connected computers, SVC offers virtual disks as ordinary SCSI devices. On the SAN side, SVC integrates various storage subsystems, even multivendor, and takes care of the correct block mapping between the SAN devices and the virtual disks for the computers
The above Storage virtualization: virtual view and physical view
The following points make the SVC an attractive tool for an energyefficient storage strategy:
Data migration from older to newer, more efficient systems can happen transparently.
Tiered storage enables you to use media with a smaller energy footprint while the SVC cache improves its performance.
Consolidation of the system's individual storage devices to virtual storage has the same effect—increasing storage utilization—as is shown for server virtualization.
Storage virtualization requires more effort than server virtualization, often requiring us to rethink the existing storage landscape. During consolidation, large amounts of data must be moved from the old systems to the consolidated storage system. This can become a long task that requires detailed planning. However, when done, the effect can be enormous because now storage can be assigned to systems in the most flexible way.
Virtual Tapes
Tapes are the cheapest medium on which to store data. They offer the largest storage volume at the lowest cost, which is the reason that they are the optimal backup medium. Tapes have a long latency compared to disks. This is not always a benefit. Currently, data centers are faced with a time limitation for backing up and probably restoring their data because the time frames for backups shrink, while the amount of data to back up expands. For this reason, many sites prefer large disk-based backup systems instead of tapes.
Tape virtualization might be a solution to this problem. A virtual tape server behaves just like a tape library, but a very fast one. This is made possible with internal disk arrays and a migration strategy to export to and import from real tape libraries.
Storage Overview
DAS,FC SAN, iSCSI, FCoE, NAS
Direct-attached storage (DAS) is digital storage directly attached to the computer accessing it, as opposed to storage accessed over a computer network. Examples of DAS include hard drives, optical disc drives, and storage on external drives.
FC SAN:
A fiber channel storage area network (FC SAN) is a system that enables multiple servers to access network storage devices. A storage area network enables high-performance data transmission between multiple storage devices and servers.
FC technology is essential for SAN implementation and establishes connectivity according to requirements.
Techopedia explains Fiber Channel Storage Area Network (FC SAN)
In SAN terminology, fabric is the hardware that connects PCs and servers to storage devices.
The following are basic FC SAN components:
- Platform devices
- Fabric-attached end devices, including servers, hosts and storage subsystems
- Devices connected to other SAN platforms via fabric facilities
- One or more FC nodes, including host adapters, host bus adapters, storage controllers and one or more node ports controlled at levels of FC-2 or higher
- Interconnected device fabric, including switches, hubs and bridges, and generic node port fabric references
- Topology Views: Identify internal fabric configuration for management applications requiring device data interconnection type. Display operational data, such as assigned fabric path routing. Management applications use topology views to construct maps with interconnected platform routes, connections and devices.
- Logical and Physical Topologies: Include end-to-end views. Logical topology enables interdependent platform device management, regardless of connection. Physical topology enables different device paths for performance and availability.
- Path Tuples: Fabric names and endpoint associators, like platforms, nodes and ports
- Logical components: Extents, data movers, platforms, fabric, nodes, FC and aggregators
- FC Arbitrated Loop (FC-AL) accounting is the most popular and scalable FC SAN configuration. FC-AL use expensive switched SAN fabrics, suits medium-sized configurations and is priced according to performance.
iSCSI : which stands for Internet Small Computer System Interface, works on top of the Transport Control Protocol (TCP) and allows the SCSI command to be sent end-to-end over local-area networks (LANs), wide-area networks (WANs) or the Internet. IBM developed iSCSI as a proof of concept in 1998, and presented the first draft of the iSCSI standard to the Internet Engineering Task Force (IETF) in 2000. The protocol was ratified in 2003.
Serial Attached SCSI (SAS) is a point-to-point serial protocol that moves data to and from computer storage devices such as hard drives and tape drives. SAS replaces the older Parallel SCSI (Small Computer System Interface, pronounced "scuzzy") bus technology that first appeared in the mid-1980s.
Different types of HARD disk : SCSI, ATA, SAS and SATA
Fibre Channel over Ethernet (FCoE) is a protocol specification to encapsulate Fibre Channel frames in Ethernet packets to enable unified I/O connectivity on a serverEncapsulation of Fibre Channel Frame in an Ethernet Packet.
With FCoE technology, servers, instead of having multiple discrete I/O adaptors for LAN and SAN traffic, will have a smaller number of converged network adapters (CNAs) that support both LAN and Fibre Channel SAN traffic
NAS : A Network Attached Storage (NAS) device is a storage device connected to a network that allows storage and retrieval of data from a centralised location for authorised network users and heterogeneous clients. NAS devices are flexible and scale-out, meaning that as you need additional storage, you can add on to what you have..
NAS for Small Businesses
Data is a critical asset for companies today. Without access to their data, companies may not be able to provide their customers with the level of service desired. Poor customer service, loss of sales or team collaboration problems are all examples of what can happen when information is not available to teams within a company.
Each of these issues contribute to lack of efficiency and potentially, loss of income if customers cannot wait for a data outage to be corrected.
Additionally, when it comes to data storage, small businesses find themselves faced with other issues, such as needing their storage solution to be:
- Lower cost
- Easy to operate (many small businesses do not have IT staff)
- Capable of growing alongside their storage needs
Types of Datastore, VMFS, NFS
Data Store : A data store is a repository of a set of data objects. These objects are modelled using classes defined in a database schema. A data store is a general concept that includes not just repositories like databases, but also simpler store types such as flat files etc
VMFS & NFS:
How Virtual Machines Access Storage
Virtual machines use virtual disks for their operating system, application software, and other data files. A virtual disk is stored as a VMDK file on a datastore. The virtual disk hides the physical storage layer from the virtual machine’s operating system. Regardless of the type of storage device that your host uses, the virtual disk always appears to the virtual machine as a local SCSI device. As a result, you can run operating systems that are not certified for specific storage equipment, such as SAN, in the virtual machine.
When a virtual machine communicates with its virtual disk stored on a datastore, it issues SCSI commands. Because datastores can exist on different types of physical storage, these commands are encapsulated into other forms, depending on the protocol that the ESX/ESXi host uses to connect to the physical storage device.
To the applications and guest operating systems running on each virtual machine, the storage subsystem appears as a virtual SCSI controller connected to one or more virtual SCSI disks as shown in the top half of Storage Architecture. These controllers are the only types of SCSI controllers that a virtual machine can see and access, and include the objects that extend
VirtualSCSIController:
ParaVirtual SCSI Controller
Virtual BusLogic Controller
Virtual LsiLogic Controller
Virtual LsiLogic SAS Controller
How precisely a virtual machine accesses storage depends on the setup of the host. Storage API Architecture gives an overview of the different possibilities.
Storage API Architecture:
Networked Storage that ESXi Supports
vSphere Features Supported by Storage
Comments
Post a Comment