Serial Attached SCSI explained

Serial Attached SCSI (SAS) is a computer bus technology primarily designed for transfer of data to and from computer data storage devices such as hard drives, CD-ROM, DVD, tape drives and similar devices. SAS is a serial communication protocol for direct attached storage (DAS) devices. It is designed for the corporate and enterprise market as a replacement for parallel SCSI, allowing for much higher speed data transfers than previously available, and is backwards-compatible with SATA drives. SATA drives may be connected to SAS controllers, but SAS drives may not be connected to SATA controllers. Though SAS uses serial communication instead of the parallel method found in traditional SCSI devices, it still uses SCSI commands for interacting with SAS End devices.

The SAS protocol is developed and maintained by the T10 technical committee of the International Committee for Information Technology Standards (INCITS) and promoted by the SCSI Trade Association (SCSITA).

Introduction

A typical Serial Attached SCSI system would consist of the following basic components:

1. An Initiator is a device that originates device service and task management requests to be processed by a target device and receives responses for the same requests from other target devices. Initiators may be provided as an on-board component on the motherboard (as is the case with many server-oriented motherboards) or as an add-on host bus adapter.
2. A Target is a device containing logical units and target ports that receives device service and task management requests for processing and sends responses for the same requests to initiator devices. A target device could be a hard disk or a disk array system.
3. A Service Delivery Subsystem is the part of an I/O system that transmits information between an initiator and a target. Typically cables connecting an initiator and target with or without expanders and backplanes constitute a service delivery subsystem.
4. Expanders are devices that are part of a service delivery subsystem and facilitate communication between SAS devices. It facilitates connection of multiple SAS End devices to a single initiator port.

SAS Domain & WWN (World Wide Name)

A "SAS Domain" is an I/O system consisting of a set of SAS devices that communicate with one another by means of a service delivery subsystem. Each SAS device in a SAS domain has a globally unique identifier assigned by the device manufacturer (similar to an Ethernet device's MAC address) called a World Wide Name (aka SAS address). The WWN uniquely identifies the device in the SAS domain just as a SCSI ID identifies a device in a parallel SCSI bus. A SAS domain may contain up to a total of 16,256 devices.

SAS (Serial Attached SCSI) vs parallel SCSI

• The SAS bus is point-to-point while the SCSI bus is multidrop. Each SAS device is connected by a dedicated link to the initiator, unless an expander is used. If one initiator is connected to one target, there is no opportunity for contention; with parallel SCSI, even this situation could cause contention.
• SAS has no termination issues and does not require terminator packs like parallel SCSI.
• SAS eliminates clock skew.
• SAS supports up to 16,384 devices through the use of expanders while Parallel SCSI is limited to 8, 16, or 32 devices on a single channel.
• SAS supports a higher transfer speed (1.5, 3.0 or 6.0 Gbit/s) than most parallel SCSI standards. The speed is realized on each initiator-target connection, hence higher throughput whereas in parallel SCSI the speed is shared across the entire multidrop bus.
• SAS controllers are required by the standard to support SATA devices.
• Both SAS and parallel SCSI use the SCSI command-set.

SAS vs SATA

• SATA devices are uniquely identified by their port number connected to the Host bus adapter while SAS devices are uniquely identified by their World Wide Name (WWN).
• Most SAS drives provide Tagged Command Queuing, while most newer SATA drives provide Native Command Queuing, each of which has its pros and cons.
• SATA follows the ATA command set and thus only supports hard drives and CD/DVD drives. In theory, SAS also supports numerous other devices including scanners and printers. However, this advantage could also be moot, as most such devices have also found alternative paths via such buses as USB, IEEE 1394 (FireWire), and Ethernet.
• SAS hardware allows multipath I/O to devices while SATA (prior to SATA II) does not. Per specification, SATA II makes use of port multipliers to achieve port expansion. Some port multiplier manufacturers have implemented multipath I/O leveraging port multiplier hardware.
• SATA is marketed as a general-purpose successor to Parallel ATA and is now common in the consumer market, while the more expensive SAS is marketed for critical server applications.
• SAS error recovery and reporting utilize SCSI commands which have more functionality than the ATA SMART commands used by SATA drives.
• SAS uses higher signaling voltages (800-1600 mV TX, 275-1600 mV RX) than SATA (400-600 mV TX, 325-600 mV RX). When SAS is mixed with SATA, the SAS drives run at SATA-voltages. One reason for this higher voltage is so SAS may be used in server backplanes.
• Because of its higher signaling voltages, SAS can use cables up to 8 m (25 ft) long, SATA is limited to 1 m (3 ft).

Characteristics

Technical details

The Serial Attached SCSI standard defines several layers (in order from highest to lowest):

• Application
• Transport
• Link
• PHY
• Physical

Serial Attached SCSI is composed of three transport protocols:

• Serial SCSI Protocol (SSP) — Supporting SAS disk drives.
• Serial ATA Tunneling Protocol (STP) — Supporting SATA disks.
• Serial Management Protocol (SMP) — for managing SAS Expanders.

For the Link and PHY layers, SAS defines its own unique protocol.

At the physical layer, the SAS standard defines connectors and voltage levels. Although not identical, the physical characteristics of the SAS wiring and signaling are so similar to SATA that it is unlikely that one technology will be faster than the other. SAS and SATA will probably both progress at the same rate to 3.0 Gbps, 6.0 Gbps, and 12.0 Gbps.

Architecture

SAS architecture consists of five layers:

• Physical layer:
  • Defines electrical and physical characteristics.
  • Differential signaling transmission.
  • Three connector types:
    • SFF 8482 – SATA compatible
    • SFF 8484 – up to four devices
    • SFF 8470 – external connector (InfiniBand connector), up to four devices
• PHY Layer:
  • Defines the signaling protocols.
• Link layer:
  • Three protocol types: SSP, STP, SMP
  • Handles the connections and transmits the data.
• Port layer:
  • Association of PHYs and SAS directions connected with other PHYs.
  • Selects the PHY to transmit.
  • Opens and closes connections.
• Transport layer:
  • Supports three transport protocols:
    • Serial SCSI Protocol (SSP): supports SAS drive devices
    • Serial ATA Tunneling Protocol (STP): supports SATA drives devices
    • Serial Management Protocol (SMP): SAS Expanders controller
• Application layer

Topology

An initiator may be directly connected to a target via one or more PHYs (such a connection is called a port whether it uses one or more PHYs, although the term "wide port" is sometimes used for a multi-PHY connection).

SAS Expanders

A Serial Attached SCSI Expander (SAS Expander) is a component used to facilitate communication between large numbers of SAS devices. Expanders contain two or more external expander ports. Each expander device contains at least one SAS Management Protocol target port for management and may contain SAS devices itself. For example, an expander may include a Serial SCSI Protocol target port for access to a peripheral device. An expander is not necessary to interface an SAS initiator and target but, if connected, helps a single initiator to communicate with more SAS/SATA targets. A useful analogy: an expander can be considered akin to a network hub in a network which allows multiple systems to be connected using a single switch port.

There are two different types of expander: Edge Expanders and Fanout Expanders.

• An edge expander allows for communication with up to 128 SAS addresses, allowing the SAS initiator to communicate with these additional devices. Edge expanders are the ones which can do direct table routing and subtractive routing. (A brief discussion of these routing mechanisms is below). Without a fanout expander, you can use at most two edge expanders in your delivery subsystem (because you will connect the subtractive routing port of those edge expanders together, and you can't connect any more expanders). To solve this bottleneck, you would use fanout expanders.
• A fanout expander can connect up to 128 sets of edge expanders, known as an edge expander device set, allowing for even more SAS devices to be addressed. The subtractive routing port of each edge expanders will be connected to the phys of fanout expander. A fanout expander can not do subtractive routing, it can only forward subtractive routing requests to the connected edge expanders.

Direct routing allows a device to identify devices directly connected to it. Table routing is for identifying devices connected to the expanders connected to a device's own PHY. Subtractive routing is used when you are not able to find the devices in the sub-branch you belong to. This will pass the request to a different branch altogether.

Expanders exist to allow more complex interconnect topologies. Expanders assist in link-switching (as opposed to packet-switching) end devices (initiators or targets). They may locate an end device either directly (when the end device is connected to it), via a routing table (a mapping of end device IDs and the expander the link should be switched to 'downstream' to route towards that ID), or when those methods fail, via subtractive routing: the link is routed to a single expander connected to a subtractive routing port. If there is no expander connected to a subtractive port, the end device cannot be reached.

Expanders with no PHYs configured as subtractive act as fanout expanders and can connect to any number of other expanders. Expanders with subtractive PHYs may only connect to two other expanders at a maximum, and in that case they must connect to one expander via a subtractive port and the other via a non-subtractive port.

There exists one root (most 'upstream') node in a SAS domain. This node is the expander which is not connected to another expander via a subtractive port. Therefore, if a fanout expander exists in the configuration, it must be the domain's root node. The root node knows about all end devices connected to the domain.

Connectors

The SAS connector is much smaller than traditional parallel SCSI connectors allowing for the small 2.5 inch drives. SAS supports point data transfer speeds up to 3 Gbit/s, but is expected to reach 12 Gbit/s by the year 2012.

The physical SAS connector is available in several different variants:

Image	Codename	Also known as	Ext/int	of pins	of devices	Comment
	SFF 8482	SATA connector	Internal		1	Form factor compatible with SATA: allows for SATA drives to connect to a SAS backplane, which obviates the need to install an additional SATA controller just to attach a DVD-writer, for example. Note that SAS drives are not usable on a SATA bus and have their physical connector keyed to prevent any plugging into a SATA backplane. The pictured connector is a drive side connector.
	SFF 8484		Internal	32 (19)	4 (2)	Hi-density internal connector, 2 and 4 lane versions are defined by the SFF standard
	SFF 8485					Defines SGPIO (extension of SFF 8484) - a serial link protocol used usually for LED indicators
	SFF 8470	Infiniband connector	External	32	4	Hi-density external connector (also used as an internal connector)
	SFF 8087	Internal mini-SAS	Internal		4	Molex iPASS reduced width internal 4x connector with future 10 Gbit/s support
	SFF 8088	External mini-SAS	External	32	4	Molex iPASS reduced width external 4x connector with future 10 Gbit/s support

See also

• List of device bandwidths

External references

• T10 committee
• SCSI Trade Association
  • Implementing SAS storage