Thursday, October 25, 2007

Windows Entropy

If you've used a Windows installation for an extended period, six months or more, then you may have noticed the system gets slower and slower as time passes. This is what I call Windows Entropy.

The main cause is the New Technology File System, or NTFS.

When you first install Windows, or format a new NTFS partition, a few things are created. The two that matter here are:
  • The Master File Table, or MFT, which is usually contiguous, reserved space for storing file metadata, like location, size, acls, and timestamps.
  • The unreserved space, where data is placed.
Initially, the data and metadata are close together on the disk. As the data grows to fill the available space, the distance between the metadata and data grows because Windows will try to keep the MFT contiguous and in one spot [source]. This distance increases seek time, which slows the whole system down because metadata is accessed frequently.

What makes it worse is the partition does not have to fill up to much cause this. If you delete a file and create a new same size or smaller file, Windows may not place the new file in the space of the old file. The data may be placed instead sparsely [source]. This further increases the distance between the data and the MFT. Other file systems do this as well because it avoids fragmentation by giving files room to grow, which is exactly why Microsoft implemented it in NTFS5. These other systems do not try to store their metadata in one contiguous space though. Ext2, for instance, stores its metadata in inodes, which reside in block groups, which are placed throughout the partition [source].