Understanding Windows Storage Stack

Understanding Windows Storage Stack

Today we are going to understand one of the most important concepts of Windows Infrastructure, which is Windows Storage Stack. After going through this website you will have a clear understanding of how your data traverse through your keyboard and get saves inside a file which is let’s say on a storage disk presented from SAN or a Local Disk.

 

For Advance users who have a very clear understanding of windows, please refer to the Article: https://blogs.technet.microsoft.com/bulentozkir/2015/04/10/windows-disk-storage-stack/ as it has a very detailed chart of Windows Storage Stack.

 

For people who are new to this, I will recommend you to go through this Article:

Understanding Windows Storage Stack

 

Storage Stack starts with the View/Medium which is visible to you. Let’s say an Application which you are using for example Notepad or Microsoft Excel. When you are entering the data, you are doing it on the Application layer.

As soon as you hit save, the data is going to traverse from the Application Layer to the IO Subsystem.

I/O Subsystem:

I/O Subsystem is a Layers as well as a Medium which is used by Windows Operation System to traverse between the various layers on Storage Stack.

  • The application sends the data to the I/O Subsystem using a specific Request Packet which goes with the Name IRP ( IO Request Packet)
  • The design allows an individual application thread to manage multiple I/O requests concurrently.
  • An IRP is a data structure that contains information completely describing an I/O request.
  • The I/O manager creates an IRP in memory to represent an I/O operation, passing a pointer of the IRP to the correct file system driver. For example, if you have an NTFS Volume where you are saving the file. The File System Driver will be NTFS.sys.

File System Driver:

Introduction

As per the Microsoft Article: What Is a File System Filter Driver?. A file system filter driver is an optional driver that adds value to or modifies the behaviour of a file system. A file system filter driver is a kernel-mode component that runs as part of the Windows executive. In easier words, all the Tasks which are associated with a volume, i.e. creating a volume, formatting a volume, saving the data on a specific location on the disk inside a volume is handled by the File system Driver. For every volume type, we have a different driver. For example for NTFS Volume we have NTFS.sys. For a REFS Volume, we have refs.sys, etc. The whole purpose of having this driver is because they know what a volume is and how it needs to handle it.

  • So Now let’s get back to our Stack. When we click save option, Our application only knows the details of the File, so it shares that information to the File System Driver. Now this driver knows the exact location of the file on the volume so it receives a Byte Offset of a File and it translates it into Volume Relative Byte offset
  • This Volume Relative Byte Offset is transferred back to the I/O Manager.
  • Now the IO Manager Takes the Request Further to the driver which has an understanding of the Volume and can understand “Volume Relative Byte offset”.

Before this happens, sometimes we require some commands to be sent to the Filesystem driver directly. For example, while taking a backup, to maintain the integrity we want the file System driver to flush all the Pending IO to the Disk. To do so we attach special-purpose drivers between the Volume Manager driver and the File System Driver.

File System Filter Drivers:

As the name suggests. These are the Filter drivers who have a File System Access and can View as well as Modify the Data at the File System Level. These drivers can be very helpful in Replication, Encryption as well as Backup operations to be handled properly.

Volume Snapshot Filter Driver:

Volsnap is a storage filter driver that resides between file system drivers and volume manager drivers (the drivers that present views of the sectors that represent a volume) so that the I/O system forwards it I/O operations directed at a volume.

Bitlocker Filter Driver:

The BitLocker filter driver (%SystemRoot%\System32\Drivers\Fvevol.sys), a kernel-mode filter driver that performs on-the-fly encryption and decryption of the volume.

 

Understanding Windows Storage Stack

  • Now since we know the drivers that can play with the Data as file system level we will move further in the Storage stack. Now since we now have a  “Volume Relative Byte offset”, we need to move this to a driver who can understand it.
  • The name of this driver is Volmgr.sys (Volume Manager Driver)

Volume Manager (Volmgr.sys)

  • The volume Manager driver is a kernel-mode driver which is used to Manage Simple Volumes which are associated with the Windows Operating System.
  • For each volume, the volume manager creates a device object of the form \Device\HarddiskVolumeX, in which X is a number (starting from 1) that identifies the volume.

 

Understanding Windows Storage Stack

 

Now the IO Manager is used to Move the  “Volume Relative Byte offset” to the Volume manager which in turn shares the request Back to I/O Manager specifying the Disk No. and Offset where the Data needs to be Written on the Disk.

Partition Manager (Partmgr.sys):

  • Partition Manager is another Layer of Storage stack that is used for discovering, deleting, creating, and managing Partitions. 
  • Now Partition manager is used to understanding if the Disk is Formatted in GPT Partition Style or MBR style partition.
  • As the IRP passes through, the time is marked. The delta between this exit time and the returned time from the disk are the metrics used in physical disk counters within Perfmon

Disk Driver ( Disk.sys ):

Disk Driver is a Class driver that implements functionality common to all disks. It understands the IRP that comes to it and it knows how to translate that IRP to the Packet which is known to the Driver that has the direct access of the Disk. These disks could be SATA attached drives or ATA Attached drivers. All these platforms have their drivers and own pattern with which they understand the IO.

  • I/O Manager sends the Data that it received from Volume Manager to the Disk Driver.
  • This Disk Driver Call the next driver to write data to disk at disk relative byte offset.

Understanding Windows Storage Stack

Port Driver:

The Last driver in any stack which is directly associated with a Port is known as the Port Driver. It will be different for different kind of Port. For example for ATA port we have ataport.sys, for SCSI Port we have SCSIport.sys and storport.sys.

People prefer to use Storport.sys over SCSIPORT because of its Performance:

For more information please refer to: https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/storport-driver-overview

The Most important function of SCSIPort.sys is to Convert the IRP to SRB (SCSI Request Block) which can be understood by the Storage controller.

Once the data is converted to the SRB, it is moved to the Storage and the data is written on the Disk.  

Understanding Windows Storage Stack

To Understand more about Disk Performance please refer to : https://knowitlikepro.com/test-storage-performance-using-diskspd/

Ashutosh Dixit

I am currently working as a Senior Technical Support Engineer with VMware Premier Services for Telco. Before this, I worked as a Technical Lead with Microsoft Enterprise Platform Support for Production and Premier Support. I am an expert in High-Availability, Deployments, and VMware Core technology along with Tanzu and Horizon.

This Post Has One Comment

Leave a Reply