The Storage Team at Microsoft - File Cabinet Blog

Data Classification Toolkit for Windows Server 2008 R2-Now Available

Nir Ben-Zvi — Wed, 17 Aug 2011 16:46:00 GMT

Identify, classify, and protect data across targeted file servers in your organization

The Solution Accelerators team is pleased to announce that the Data Classification Toolkit for Windows Server 2008 R2 is now available for download.

Download the toolkit

The Data Classification Toolkit for Windows Server 2008 R2 is designed to help enable an organization to identify, classify, and protect data on their file servers. The out-of-the-box classification and rule examples help organizations* build and deploy their policies to protect critical information in a cost-effective manner

Data Classification Toolkit for Windows Server 2008 R2 Beta Now Available

Nir Ben-Zvi — Wed, 01 Jun 2011 03:07:00 GMT

Consistently identify, classify, and protect data across all file servers in your organization

The Solution Accelerators team is pleased to announce the Data Classification Toolkit for Windows Server 2008 R2 Beta.

Join the Beta Program (Windows Live ID required)

Designed to help reduce the cost and complexity of data compliance, the Data Classification Toolkit for Windows Server 2008 R2 helps organizations consistently identify, classify, and protect data across multiple file servers. Using out-of-the-box classification knowledge, this tool gives organizations visibility into how data is distributed across their file servers to help them apply the right policies, protect critical data, and identify potential data storage efficiencies

Want to learn more about IT Governance and Compliance? Visit the IT GRC home page, and check out all the free content the IT GRC team has to offer!

Microsoft IT Uses File Classification Infrastructure to Help Secure Personally Identifiable Information

Nir Ben-Zvi — Wed, 25 May 2011 06:20:00 GMT

I'd like to bring to your attention a recently published Technical Case Study from Microsoft IT that showcases how Microsoft IT is using FCI.

Learn how Microsoft Information Technology (IT) used File Classification Infrastructure (FCI) to create a solution to automatically classify, manage, and protect sensitive data, including personally identifiable information and financial information. Using the new FCI-based solution, Microsoft IT can obtain file-level details about content sensitivity while reducing misclassification of personally

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=bee97542-c6c6-45b9-88c4-3abfdbb92e38

TechEd 2011 demo install step-by-step (Hyper-V, AD, DNS, iSCSI Target, File Server Cluster, SQL Server over SMB2)

Jose Barreto - MSFT — Sat, 21 May 2011 15:11:00 GMT

We have a new blog post out that explains the demo setup used in the TechEd 2011 presentation about Windows Server 2008 R2 File Services Consolidation.

In the post, you get step-by-step instructions on how to reproduce the environment used in the presentation's demo. This is a great way to experiment with a fairly large set of Microsoft technologies, including:

Windows Server 2008 R2
Hyper-V
Networking
Domain Name Services (DNS)
Active Directory Domain Services (AD-DS)
iSCSI Software Target 3.3
iSCSI Initiator
File Server (SMB2)
Failover Clustering (WSFC)
SQL Server 2008 R2

This is a long post with dozens of screenshots and it packs a lot of information. Check it out at

http://blogs.technet.com/b/josebda/archive/2011/05/19/teched-2011-demo-install-step-by-step-hyper-v-ad-dns-iscsi-target-file-server-cluster-sql-server-over-smb2.aspx

File Server Team sessions at TechEd 2011 this week

Jose Barreto - MSFT — Mon, 16 May 2011 11:29:00 GMT

If you're attending TechEd 2011 this week, here are sessions from the File Server team:

WSV313 - Microsoft iSCSI Software Target 3.3 for Application Storage, Diskless Boot, and More!
Speaker(s): Jian (Jane) Yan
Tuesday, May 17 at 5:00 PM, room: B101
http://northamerica.msteched.com/topic/details/WSV313

WSV317 - Windows Server 2008 R2 File Services Consolidation: Technology Update
Speaker: Jose Barreto
Wednesday, 5/18 at 10:15am, room: Georgia Ballrm 3
http://northamerica.msteched.com/topic/details/WSV317

WSV317-R - Windows Server 2008 R2 File Services Consolidation: Technology Update (repeat)
Speaker: Jose Barreto
Thursday, 5/19 at 10:15am, Room: Georgia Ballrm 2
http://northamerica.msteched.com/topic/details/WSV317-R

WSV318 - Windows Storage Server 2008 R2 Technical Overview
Speakers: Joel Garcia, Scott M. Johnson
Wednesday, May 18 at 3:15 PM, room: B309
http://northamerica.msteched.com/topic/details/WSV318

WSV323 - Information Governance for Unstructured Data Using the Data Classification Toolkit for Windows Server 2008 R2
Speakers: Gunjan Jain, Nir Ben Zvi
Wednesday, May 18 at 10:15 AM, room: C206
http://northamerica.msteched.com/topic/details/WSV323

Also make sure to visit the Windows Server booth for File Services (WSV 13).

Using 4k sector and advanced format drives in Windows. HotFix and support info for Windows Server 2008 R2 and Windows 7

Jose Barreto - MSFT — Tue, 26 Apr 2011 21:37:00 GMT

If you work with storage, you probably already heard about the “4K Sector Drives”, “Advanced Format Drives” and “512e drives”. These new “4K sector drives” abandon the traditional use of 512 bytes per sector in favor of a new structure that uses 4096 bytes. The migration to the new formats is eased by the use of 4K drives that simulate the old format, known as “512 Emulation Drives” or “512e Drives” or Advanced Format Drives”.

Native 4K sector drives are currently not supported with Windows. However, 512e drives (or Advanced Format Drives) are supported with recent versions of Windows, provided that you follow the guidance in the following support article: http://support.microsoft.com/kb/2510009. There are specific requirements to be met and specific details for different Microsoft applications like Hyper-V, SQL Server and Exchange Server.

For Windows 7 and Windows Server 2008 R2, the KB article above mentions the requirement to install a specific hotfix described at http://support.microsoft.com/kb/982018. Please note that most of this fix is part of Windows 7 Service Pack 1 (SP1) or Windows Server 2008 R2 SP1, except for updates to the FSUTIL tool.

For you developers, head on over to MSDN to read on the nitty gritty details of this storage transition, and how it may impact your applications. Details are published at http://msdn.microsoft.com/en-us/library/hh182553.aspx.

If you’re interested in these new 4K sector drives, you might also want to look at these other links:

Note: The updated version of FSUTIL, is available as a download from the support KB page and, since 4/26/2011, via Windows Update labeled as "Update for Windows 7 (KB982018)".

FSCT test results detail the performance of Windows Server 2008 R2 File Server configurations - 23,000 users with 192 spindles

Jose Barreto - MSFT — Mon, 11 Apr 2011 15:48:00 GMT

The File Server Capacity Tool (FSCT) is a free download from Microsoft that helps you determine the capacity of a specific file server configuration (running Windows or any operating system that implements the SMB or SMB2 protocols). It simulates a specific set of operations (the “Home Folders” workload) being executed by a large number of users against the file server, confirming the ability of that file server to perform the specified operations in a timely fashion. It makes it possible to verify, for instance, if a specific file server configuration can handle 10,000 users. In case you’re not familiar with FSCT’s “Home Folders Workload”, it simulates a standard user’s workload based on Microsoft Office, Windows Explorer, and command-line usage when the file server is the location of the user’s home directory. We frequently use FSCT internally at Microsoft. In fact, before being released publicly, the tool was used to verify if a specific change to the Windows code has any significant performance impact in a file server scenario. We continue use FSCT for that purpose today.

Recently, the File Server Team released a document with results from a series of FSCT tests. These tests were performed in order to quantify the file server performance difference between Windows Storage Server 2008 (based on Windows Server 2008) and Windows Server 2008 R2. It was also an exercise to analyze the capacity (in terms of FSCT “Home Folders” users) of some common File Server configurations using between 24 and 192 disks. The 192-spindle configuration was able to handle 23,000 FSCT users running the Home Folders workload. Check the blog post at http://blogs.technet.com/b/josebda/archive/2011/04/08/fsct-test-results-detail-the-performance-of-windows-server-2008-r2-file-server-configurations-23-000-users-with-192-spindles.aspx for further details and a link to the document in the Microsoft Download Center.

Microsoft iSCSI Software Target 3.3 for Windows Server 2008 R2 available for public download

Jose Barreto - MSFT — Mon, 04 Apr 2011 19:27:00 GMT

Today, Microsoft has made the Microsoft iSCSI Software Target 3.3 publicly available to all users of Windows Server 2008 R2. The Microsoft iSCSI Software Target has been available for production use as part of Windows Storage Server since early 2007. It has also been available for development and test use by MSDN and TechNet subscribers starting in May 2009. However, until now, there was no way to use the Microsoft iSCSI Software Target in production on a regular server running Windows Server 2008 R2. This new download offers exactly that. Get all the details, including download instructions and FAQ at http://blogs.technet.com/b/josebda/archive/2011/04/04/microsoft-iscsi-software-target-3-3-for-windows-server-2008-r2-available-for-public-download.aspx

Windows Server Backup automatic disk usage management

Jose Barreto - MSFT — Mon, 14 Mar 2011 18:17:00 GMT

This blog applies to Automatic Disk Usage management and Version Management for versions lying on local disk. This does not apply to versions lying on Network shares.

Windows Server Backup Overview

Windows Server Backup is the built-in backup solution in Windows Server 2008 and Windows Server 2008 R2. By using Windows Server Backup, an administrator can schedule periodic backups of a server and also create backups on demand. For details on using Windows Server Backup, please see the Installed Help for Windows Server 2008 (http://technet.microsoft.com/en-us/library/cc770593(WS.10).aspx) and for Windows Server 2008 R2 (http://technet.microsoft.com/en-us/library/cc770757.aspx).

Windows Server Backup stores backup versions in volume shadow copies. After the data write is complete, Windows Server Backup creates a shadow copy of the volume where the backup is stored using Volume Shadow Copy Service (VSS). This shadow copy retains the state of the storage volume as a “backup version” or “point-in-time” of the backup and must restore using this backup version. VSS is the underlying Microsoft technology required for maintaining backup versions. (For more information about VSS, see http://technet.microsoft.com/en-us/library/cc785914.aspx)

This article answers the following questions with regards to Automatic Disk Usage management for Windows Server Backup in Windows Server 2008 and Windows Server 2008 R2:

Q1. What is Automatic disk usage management feature in Windows Server Backup?
Q2. How does the auto-delete functionality of Windows Server Backup works?
Q3. What are the criteria on which this feature works?
Q4. Why does backup fail with target out of disk space in spite of automatic disk usage management feature in Windows Server Backup?

Automatic disk usage management

After you configure a disk for a scheduled backup, Windows Server Backup automatically manages the disk usage—you do not need to be concerned about running out of disk space after repeated backups. Windows Server Backup will automatically reuse the space occupied by older backups when creating new backups. The management tool displays the backups that are available as well as the disk usage information. This can help you plan for provisioning additional storage to meet your recovery objectives.

How does it work?

Automatic Disk usage management feature comes into play when Windows Server Backup detects that the backup target does not have enough space to accommodate the backup while backup is in progress. The way Windows Server Backup creates space for new backup is by shrinking the storage space allocated for snapshots (called diff area). As a result, one or more older snapshots (and hence backup versions corresponding to those snapshots) occupying the diff area that got shrunk get deleted. Before shrinking diff area, WSB determines whether shrinking the diff area can free up the requisite space so the backup can happen. If enough free space can get created, WSB goes ahead with the shrinking and continues with the backup. WSB will not shrink the diff area to less than 1/8 of Target volume size as we do not want to lose all past backups just to accommodate this one. This is why sometimes backup fail with target out of disk space in spite of automatic disk usage management feature in Windows Server Backup.

Criteria:

This feature will not work if Windows Server Backup has reached the state where it cannot further contract the snapshot allocated space and also preserve some of the existing backups. This happens when space required to complete the backup causes the snapshot storage space to become less than 1/8 of the target size. You can change the snapshot storage from here: Volume à properties àshadow copies àsettings.

If the Automatic Disk usage management criterions are not satisfied, you can still delete old backups created by Windows Server Backup. For the steps to do this, refer to the next section.

Backup Version Management

If you want to delete the system state backup, you may use "wbadmin delete systemstatebackup" in command shell.

Windows Server Backup does not support backup deletion of other kinds. However, there is a work around if you really need to delete backups on a target. You can delete the shadow copies where the backup exists. Note that these steps are not recommended and utmost care must be taken to not delete backups incorrectly. If you incorrectly delete your backups, these will be permanently lost.

The steps to delete the backups are as below,

Get the shadow copy Id for a backup using the command "wbadmin get versions -backuptarget:<TargetDrive>". This command will list all the backups on the given target with their shadow copy IDs.
If you delete all consecutive shadow copies (that correspond to backups you no longer need) starting from the oldest shadow copy, you can get some space. Deleting backup versions in middle would not guarantee that free space gets created. This is because of the inherent limitation of snapshot technology on which WSB relies.

Detailed information on Backup Version Management:

http://blogs.technet.com/b/filecab/archive/2009/06/22/backup-version-and-space-management-in-windows-server-backup.aspx

Post by Mithilesh Singh

File Server improvements from Windows Server 2003 to Windows Server 2008 R2. 8 items for 8 years…

Jose Barreto - MSFT — Sat, 26 Feb 2011 06:17:00 GMT

A new blog post covers a set of Windows Server 2003 File Server problems that were fixed in Windows Server 2008 and Windows Server 2008 R2. Obviously a lot has changed in Windows Server File Services in the last 8 years, but the post highlights 8 specific issues commonly mentioned by users that are still running Windows Server 2003. Here's the list:

CHKDSK Time
File Server Scalability
Failover Cluster Setup
Shares Per File Server
Encrypted Files
Large File Transfers on High Latency Networks
Transferring Lots of Small Files
Caching in the Branch

Get all the details about exactly how Windows Server 2008 and Windows Server 2008 R2 address all these issues and how much changed in SMB and File Services.

Read the full post at http://blogs.technet.com/b/josebda/archive/2011/02/25/file-server-improvements-from-windows-server-2003-to-windows-server-2008-r2-8-items-for-8-years.aspx

New white paper providing guidance for sizing NTFS volumes

Jose Barreto - MSFT — Wed, 23 Feb 2011 01:28:00 GMT

Introduction

Today, Microsoft released a new white paper, available here, discussing best practices and guidance for sizing NTFS volumes and some details on CHKDSK execution times on Windows Server 2008 R2.

When planning Windows file server deployments, we are often asked questions such as “How large can I make my volumes?” or “How long will it take to repair a volume?”. This white paper helps answer these questions.

NTFS self-healing

NTFS self-healing is a feature introduced in Windows Vista and in Windows Server 2008 to reduce the need to run CHKDSK. NTFS self-healing attempts to fix certain corruptions encountered during normal operation, which significantly reduces the frequency of NTFS marking the volume dirty. When NTFS can self-heal, it will not have to perform CHKDSK on the volume during system startup. Self-healing fixes corruptions while the volume is still online, which increases the overall availability of the systems.

CHKDSK improvements

CHKDSK was updated in Windows 7 and Windows Server 2008 R2 to improve how it caches metadata, also known as Block Cache. A significant portion of a CHKDSK run is the scan phase, where it seeks different locations on disk to verify information from file system metadata objects. By caching larger blocks of the disk in RAM, CHKDSK execution time is reduced by reducing disk seeks. The Block Cache feature also reduces the need to re-access data from the disk, reducing the total device I/O time, and therefore, the overall CHKDSK execution time. The Block Cache feature may cause CHKDSK to consume a significant amount of memory, which is not usually a concern in Windows file servers, but reduces the amount of disk I/O and therefore significantly improves performance. The net result is greatly reduced execution time in common scenarios.

15 TB NTFS file systems with 10 million files

A common misconception that I often run into is that NTFS file system does not support large volumes. I have heard customers quote volume sizes of 300-500GB as the perceived maximum volume size for NTFS file systems. In the first test, the team stood up a machine with 5, 10 and 15 TB volumes and placed 10 million files on each volume. This demonstrates that it is certainly possible to stand up relatively large file systems with NTFS, debunking the myth of NTFS only being able to scale to 300-500GB volumes.

How long does it take to run CHKDSK?

Another commonly raised question is “how long does it take to complete CHKDSK?”. The team performed multiple CHKDSK runs on the volumes described above and observed the following data:

Volume size (in TB's)	Time (sec)	Time (mins)
5	384	6.4
10	407	6.9
15	417	7.0

This demonstrates that CHKDSK execution time is not significantly affected by size of the volume. CHKDSK execution time is a function of the number of files in the NTFS file system, as demonstrated in another test performed by the team and described in the white paper. The other observation is that CHKDSK execution time for a NTFS file system with 10 million files is measured in minutes (~7 minutes) as opposed to hours or days, which was sometimes the case with older versions of Windows.

Conclusion

If you are planning a Windows file server deployment or is looking to upgrade an existing Windows file server deployment to Windows Server 2008 R2, you should consult this white paper. It demonstrates that with Windows Server 2008 R2:

1. Windows NTFS can support 15 TB file systems, each with 10 million files

2. CHKDSK execution time is a matter of minutes, not hours or days

Post by Claus Joergensen

Two new blog posts on Windows Storage Server 2008 R2

Jose Barreto - MSFT — Fri, 19 Nov 2010 07:45:50 GMT

Scott M. Johnson has published two new blogs post on the Windows Storage Server blog:

The first post is a compilation of links covering all kinds of Windows Storage Server 2008 R2 content.
http://blogs.technet.com/b/storageserver/archive/2010/11/16/windows-storage-server-2008-r2-references.aspx

The second post introduces the two new offerings from HP (X1000 G2 and X3000 G2) that were introduced during the recent TechEd Europe event, which run the new Windows Storage Server 2008 R2.
http://blogs.technet.com/b/storageserver/archive/2010/11/16/hp-rocks-teched-with-their-new-windows-storage-server-product-line.aspx

Enjoy...

Windows Storage Server 2008 R2 and the Microsoft iSCSI Software Target 3.3 are available on MSDN/TechNet - here's how to install them

Jose Barreto - MSFT — Tue, 28 Sep 2010 06:16:00 GMT

You probably heard that Windows Storage Server 2008 R2 has been Released To Manufacturing (RTM) last week. That means that the OEMs will soon deliver a new generation of Windows-based NAS appliances with a file server (SMB2, NFS) and an iSCSI Target (using the Microsoft iSCSI Software Target 3.3).

What you might have missed is that Windows Storage Server 2008 R2 is also now available to MSDN and TechNet subscribers for evaluation. This means subscribers can now use the Microsoft iSCSI Target 3.3 for Development, Test and all other non-production purposes. While this was also the case for Windows Storage Server 2008, this new R2 release allows you to install the iSCSI Target on any existing Windows Server 2008 R2 by simply installing an MSI (Microsoft Installer) file. That’s much more convenient for Developers, Testers and IT Profressionals out there.

Find all the details about how to download from MSDN/TechNet, unpack and install in this new blog post: http://blogs.technet.com/b/josebda/archive/2010/09/27/windows-storage-server-2008-r2-and-the-microsoft-iscsi-software-target-3-3-are-available-on-msdn-technet-here-s-how-to-install-them.aspx

Windows Storage Server 2008 R2 is now available!

Jose Barreto - MSFT — Thu, 23 Sep 2010 00:37:53 GMT

It's a pleasure to let you know that Windows Storage Server 2008 R2 (which includes the Microsoft iSCSI Software Target 3.3) has been released to manufacturing (RTM).

Scott M. Johnson (Program Manager in the File Server Team) has just shared via a blog post that the product is on its way to the OEMs and included quite a few details, including:

Overview
Three New Editions
Key Scenarios
- File Server
- Branch Office Server
- Block Storage Server
- Unified Storage Server
- iSCSI Boot Server
- iSCSI Boot for HPC Clusters
- Gateway to a SAN
Deployment Modes
- Standalone Storage Server
- Highly Available Storage Server
iSCSI Topologies Tested at Microsoft
- Hyper-V host using iSCSI LUN as a volume with VHD files
- Hyper-V host using iSCSI LUN as pass-through disk
- Hyper-V guest using iSCSI LUN directly
- Boot and Data disks for a Hyper-V host
- Clustered Application Servers
- Clustered Hyper-V host for Live Migration and CSV
Components Unique to Windows Storage Server
- Single Instance Storage (SIS)
- New! Cluster-Ready OOBE
- Web RDP Management
- iSCSI Software Target 3.3
- iSCSI Software Target 3.3 Hardware Providers
- Branding and Licensing Packages

Scotts wraps it up with a nice table comparing the 3 editions of Windows Storage Server 2008 R2.

Check all the details at http://blogs.technet.com/b/storageserver/archive/2010/09/22/windows-storage-server-2008-r2-is-now-available.aspx.

Congrats to the team for a job well done!

New five-part blog series on DFS Replication

Jose Barreto - MSFT — Tue, 07 Sep 2010 15:37:51 GMT

Ned Pyle, a Senior Escalation Support Engineer with the Directory Services team at Microsoft, has just started a new blog series focusing on DFS Replication.

Here’s how he introduces it:

Hello folks, Ned here again to kick off a new five-part series on DFSR. With the release of Windows Server 2008 R2, the warming of economies, and the timing of hardware leases, we have started seeing more questions around replacing servers within existing DFSR Replication Groups. Through the series I will discuss the various options and techniques around taking an existing DFSR replica and replacing some or all of its servers. Depending on your configuration and budget, this can range from a very seamless operation that users will never notice to a planned outage where even their local server may not be available for a period of time. I leave it to you and your accountants to figure out which matters most. This series also gives updated steps on validated pre-seeding to avoid any conflicts and maximize your initial sync performance. I will also speak about new options you have in this replacement cycle for clusters and read-only replication.

Here’s the series index:

· Replacing DFSR Member Hardware or OS (Part 1: Planning)

· Replacing DFSR Member Hardware or OS (Part 2: Pre-seeding)

· Replacing DFSR Member Hardware or OS (Part 3: N+1 Method)

· Replacing DFSR Member Hardware or OS (Part 4: Disk Swap)

· Replacing DFSR Member Hardware or OS (Part 5: Reinstall and Upgrade)

You can read Part 1 right now at http://blogs.technet.com/b/askds/archive/2010/09/03/replacing-dfsr-member-hardware-or-os-part-1-planning.aspx

Using the multiple NICs of your File Server running Windows Server 2008 (and 2008 R2)

Jose Barreto - MSFT — Fri, 03 Sep 2010 16:52:26 GMT

New blog post covers Multi-NIC File Server configurations:

When you set up a File Server, there are advantages to configuring multiple Network Interface Cards (NICs). However, there are many options to consider depending on how your network and services are laid out. Since networking (along with storage) is one of the most common bottlenecks in a file server deployment, this is a topic worth investigating.

Throughout this blog post, we will look into different configurations for Windows Server 2008 (and 2008 R2) where a file server uses multiple NICs. Next, we’ll describe how the behavior of the SMB client can help distribute the load for a file server with multiple NICs. We will also discuss SMB2 Durability and how it can recover from certain network failure in configuration where multiple network paths between clients and servers are available. Finally, we will look closely into the configuration of a Clustered File Server with multiple client-facing NICs.

Read the full post, complete with diagrams and screenshots, at
http://blogs.technet.com/b/josebda/archive/2010/09/03/using-the-multiple-nics-of-your-file-server-running-windows-server-2008-and-2008-r2.aspx

Using Kerberos security with Server for NFS

Jose Barreto - MSFT — Thu, 13 May 2010 22:59:00 GMT

I'll discuss how to configure a UNIX based NFS client to connect to Windows Server for NFS using Kerberos security with RPCSEC_GSS.

Traditionally NFS clients and servers use AUTH_SYS security. This essentially allows the clients to send authentication information by specifying the UID/GID of the UNIX user to an NFS Server. Each NFS request has the UID/GID of the UNIX user specified in the incoming request. This method of authentication provides minimal security as the client can spoof the request by specifying the UID/GID of a different user. This method of authentication is also vulnerable to tampering of the NFS request by some third party between the client and server on the network.

RPCSEC_GSS provides a generic mechanism to use multiple security mechanisms with ONCRPC (on which NFS requests are built). Server for NFS currently provides support for two Kerberos "flavors" over NFS using RPCSEC_GSS: krb5 and krb5i. krb5 provides Kerberos authentication at the RPC request level, while krb5i (Kerberos v5 with Integrity) also protects the NFS payload from tampering.

Explaining how to set up Kerberos security between a UNIX client and a Windows server running Server for NFS can best be accomplished by way of a simple example. In this tutorial we'll consider the following infrastructure scenario:

· Windows domain called NFSDOMAIN.COM running Active Directory on a DC named nfsdomain-dc.nfsdomain.com

· Windows server running Server for NFS: windowsnfsserver.nfsdomain.com

· UNIX client: unixclient.nfsdomain.com

· UNIX user on unixclient.nfsdomain.com: unixuser1 with UID 500

· UNIX group on uinxclient.nfsdomain.com: unixgroup1 with GID 500

· Windows user (NFSDOMAIN\unixuser1)

· Windows group (NFSDOMAIN\unixgroup1)

For the purposes of this configuration guide I'll be assuming unixclient.nfsdomain.com is running OpenSolaris.

Basics:

First, make sure that DNS name resolution is working properly using between the DC, the Windows NFS Server, and the UNIX client.

One caveat for the Solaris client is that the hostname should be set to just the first part of the FQDN. Running "hostname" on unixclient.nfsdomain.com should output only "unixclient". If not, set the hostname to just "unixclient".

Make sure that all the above users and groups have been created and that NFSDOMAIN\unixuser1 is a member of NFSDOMAIN\unixgroup1 group. Be sure that you've set a password for unixuser1. If the UID/GID numbers are different on the UNIX client, just substitute the appropriate values where I'm using 500.

Joining the UNIX client to AD:

Now we're going to configure unixclient to get Kerberos tickets from the NFSDOMAIN.COM domain. This is done by editing the /etc/krb5/krb5.conf file:

There should be an existing file with some placeholders which can be edited. We're going to fill in the following fields: "default_realm", "kdc", "admin_server", and "[domain_realm]". We're also going to add two lines under "[libdefaults]" for "default_tkt_enctypes" and "default_tgs_enctypes". I'll get to why these are added later. The end result should look something like:

<Begin /etc/krb5/krb5.conf snip>
[libdefaults]
        default_realm = NFSDOMAIN.COM
        default_tkt_enctypes = arcfour-hmac-md5 des3-cbc-sha1-kd des-cbc-md5
        default_tgs_enctypes = arcfour-hmac-md5 des3-cbc-sha1-kd des-cbc-md5

[realms]
        NFSDOMAIN.COM = {

                      kdc = nfsdomain-dc.nfsdomain.com
                      admin_server = nfsdomain-dc.nfsdomain.com
        }

[domain_realm]
        nfsdomain.com = NFSDOMAIN.COM
</snip>

At this point we should be able to test getting a ticket for NFSDOMAIN\unixuser1 from unixclient:

From unixclient run: "kinit unixuser1" and type in the user's password.

Now run "klist", you should have a ticket for unixuser1! Run "kdestroy" to destroy the ticket.

How does NFS use RPCSEC_GSS?

Ok, now that we have the basic Kerberos setup working I'll explain a bit how authentication works from the NFS standpoint.

When unixclient wants to authenticate with windowsnfsserver, it needs some other "user" (in Kerberos this is called a "principal") to authenticate with. The way this happens is that when a NFS share is mounted, the client looks at the FQDN of the NFS Server and expects to authenticate with the following principal: "nfs/FQDN@domain_realm". In this case unixclient is going to look for "nfs/windowsnfsserver.nfsdomain.com@NFSDOMAIN.COM". By convention, UNIX machines do the initial few NFS operations as "root". In this context it's the root account of the local machine and again, by convention, this principal is "root/unixclient.nfsdomain.com@NFSDOMAIN.COM".

What does this mean? How do I set up these principals?

For "nfs/windowsnfsserver.nfsdomain.com@NFSDOMAIN.COM" we're just going to user the existing "machine" account in AD and essentially create an alias for it.

From nfsdomain-dc run:

setspn -A nfs/windowsnfsserver windowsnfsserver
setspn -A nfs/windowsnfsserver.nfsdomain.com windowsnfsserver

then:

setspn -L windowsnfsserver

You should see all of the machine account's "service principal names".

What about the principals for unixclient?

We're going to create some users in AD for these principals. Because "/" is not a valid character for AD account names we'll have to pick a different name, then use the setspn utility to add the service principal names. For the sake of completeness, we're going to create a couple more accounts than just the root account.

on nfsdomain-dc create the following users and set passwords for them:

· unixclienthost (represents the unixclient machine)

· unixclientroot (account for root on unixclient)

· unixclientnfs (used by the NFS Server on unixclient)

After creating these users, right click on each user, go to properties and under the "Account" tab change the "User logon name" to "host/unixclient.nfsdomain.com", "root/unixclient.nfsdomain.com", and "nfs/unixclient.nfsdomain.com" respectively.

Now we're going to set the SPNs on these accounts:

setspn -A host/unixclient unixclienthost
setspn -A host/unixclient.nfsdomain.com unixclienthost
setspn -A root/unixclient unixclientroot
setspn -A root/unixclient.nfsdomain.com unixclientroot
setspn -A nfs/unixclient unixclientnfs
setspn -A nfs/unixclient.nfsdomain.com unixclientnfs

unixclient needs to use the root/unixclient.nfsdomain.com principal without actually typing in a password for that account. This is accomplished with a "keytab" file.

We're going to export keytab files for these accounts. On nfsdomain-dc run:

ktpass -princ host/unixclient.nfsdomain.com@NFSDOMAIN.COM -mapuser unixclienthost -pass <insert password> -out unixclienthost.keytab
ktpass -princ root/unixclient.nfsdomain.com@NFSDOMAIN.COM -mapuser unixclientroot -pass <insert password> -out unixclientroot.keytab
ktpass -princ nfs/unixclient.nfsdomain.com@NFSDOMAIN.COM -mapuser unixclientnfs -pass <insert password> -out unixclientnfs.keytab

Now move these files from nfsdomain-dc to unixclient.

On unixclient we're going to merge these files in the keytab file. From the directory where the files were copied run: "ktutil". In this interactive tool run the following commands:

rkt /etc/krb5/krb5.keytab
rkt unixclienthost.keytab
rkt unixclientroot.keytab
rkt unixclientnfs.keytab
wkt /etc/krb5/krb5.keytab
q

Great, now unixclient should be able to get tickets for these accounts without typing any passwords. Test this out:

kinit -k host/unixclient.nfsdomain.com
kinit -k root/unixclient.nfsdomain.com
kinit -k nfs/unixclient.nfsdomain.com

Each of these commands should successfully get a ticket. This can be a bit tricky to get working in some configurations because the underlying encryption mechanism encoded in the ticket must be supported by both nfsdomain-dc and unixclient. You may also need to explicitly specify an encryption type when using the ktpass utility. "-crypto DES-CBC-MD5" seems to be widely supported. Depending on the version of AD in your environment, you may need to check the "Use Kerberos DES encryption types for this account" under the "Account" tab in the user property page. Check the security event log to help debug the logon attempts.

Mapping UID/GID to Kerberos principals on unixclient:

Ok, at this point we need to teach the NFS client on unixclient how to map its local UIDs and GIDs to Kerberos principals:

On unixclient run the following:

gsscred -m kerberos_v5 –a
gsscred -m kerberos_v5 –n host/unixclient.nfsdomain.com -u 0 –a
gsscred -m kerberos_v5 –n root/unixclient.nfsdomain.com -u 0 –a
gsscred -m kerberos_v5 –n nfs/unixclient.nfsdomain.com -u 0 –a
gsscred -m kerberos_v5 –n unixuser1 -u 500 –a
gsscred -m kerberos_v5 -n unixgroup1 -g 500 -a

These commands populated the GSS credentials table which is used to match incoming principals to local user accounts. Make sure that "gssd" is running on unixclient.

Enable Kerberos security for NFS:

OpenSolaris disables Kerberos over NFS by default. Obviously we need to enable it. Edit "/etc/nfssec.conf" and uncomment the "krb" lines. You should have the following:

<snip>
krb5            390003 kerberos_v5     default -               # RPCSEC_GSS
krb5i           390004 kerberos_v5     default integrity       # RPCSEC_GSS
krb5p           390005 kerberos_v5     default privacy         # RPCSEC_GSS
</snip>

Note that Server for NFS does not currently support krb5p.

Set up UID/GID mappings for Server for NFS:

On the windows side, you'll still need to configure mapping between UNIX UIDs and GIDs to windows accounts. Server for NFS will always use the Kerberos principal it receives for each NFS request. However, the NFS protocol still only returns UIDs and GIDs for requests like GETATTR and READDIR. If no mapping is set up a "ls -l" on unixclient won't show the correct UIDs and GIDs even though newly created files will have the correct security descriptor visible from windowsnfsserver.

Specify How Server for NFS Obtains Windows User and Group Information:
http://technet.microsoft.com/en-us/library/cc754514.aspx

This blog has useful information for using the Active Directory Lookup feature:
http://blogs.msdn.com/sfu/archive/tags/Active+Directory+Lookup/default.aspx

Provision a share on windowsnfsserver:

This configuration guide has details on how to set up a share:
http://technet.microsoft.com/en-us/library/cc770569.aspx

After creating the "share" directory, make sure that all intended users (including unixuser1) have access to it.

Mount the share:

Finally, we're ready to mount a share!

Remember to use the FQDN of the server when mounting. This is so unixclient can figure out the proper principal name to look up for windowsnfsserver.

As root on unixclient run:

mount -o sec=krb5,vers=3,proto=tcp windowsnfsserver.nfsdomain.com:/share /mnt/share

Once the share has been mounted log in as unixuser1 and run "kinit". Type in the password and access "/mnt/share". If everything was properly configured you should have access. Congratulations, your NFS connection is now vastly more secure!

Helpful links:

Services for NFS Step-by-Step Guide for Server 2008 R2:
http://technet.microsoft.com/en-us/library/dd758767(WS.10).aspx

Windows/UNIX authentication configuration:
http://blog.scottlowe.org/2006/08/08/linux-active-directory-and-windows-server-2003-r2-revisited/

QFE for AD encryption bug (probably not necessary if you’ve applied updates):
http://support.microsoft.com/kb/951191

OpenSolaris encryption bug:
http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6548599

Post by Jeff Biseda

File Classification Infrastructure webcasts with DataGlobal and Titus labs

Nir Ben-Zvi — Sun, 25 Apr 2010 23:36:00 GMT

File Classification – Taming the Wild Beast of Unstructured Data by Microsoft (Apr. 19, 2010)
It’s a jungle of electronically stored information that has proliferated far beyond the ability of manual data management. In fact, this unstructured data has taken over 85 percent of today’s enterprise, however, that wild data holds invaluable and untapped information for today's corporation, including intellectual property, product expertise and knowledge stores. But just like any wild animal, that same data also holds hidden dangers because of unseen regulatory violations, court case evidence, confidential employee information and trade secrets, representing a wide range of risks that must be tamed in order to protect the corporation. In addition to the potential risks, these warehouses of information also offer the single largest opportunity for increasing revenue and reducing costs. Tapping into the knowledge stored in these volumes can save money and introduce new revenue-generating opportunities while avoiding compliance violations and fines.

File Classification is critical to tame the wild beast of this massive amount of unstructured data. It delivers on the business needs of eDiscovery, compliance and knowledge reuse while enabling the company to manage its physical storage to reduce escalating hardware costs and footprint requirements. In this session you will learn how to leverage the Windows Server 2008 R2 File Classification Infrastructure (FCI) and DataGlobal Enterprise Resource Suite to deliver significantly enhanced file classification capabilities into an enterprise of any size in order to improve revenue, profitability and competitive advantage while reducing corporate risks and costs simultaneously.

Enhanced Classification and Policy Enforcement with Windows Server 2008 R2 and Titus Labs by (Apr. 19, 2010)
With the sheer volume of data on servers, it has become very difficult for IT administrators to manage the information and mitigate the risks of data leakage. File classification is a critical first step in addressing these challenges. This session covers how to leverage Windows servers to deliver enhanced classification and policy enforcement.

In this session you will learn how to leverage Windows servers, Microsoft has introduced the Windows Server 2008 R2 File Classification Infrastructure (FCI) and Titus Labs solution and deliver enhanced classification and policy enforcement.

Using File Classification Infrastructure (FCI) and AD RMS to automatically protect sensitive information

Nir Ben-Zvi — Sun, 31 Jan 2010 05:12:00 GMT

Windows Server 2008 R2 File Classification Infrastructure provides a built-in solution for file classification allowing administrators to automate manual processes with predefined policies based on the data’s business value. (Windows Server 2008 R2 File Classification Infrastructure – Managing data based on business value)

The Active Directory Rights Management Services Bulk Protection Tool provides a command-line interface that you can use to decrypt AD RMS protected files. You can also use it to encrypt multiple files to a predefined rights policy template.

Together, the AD RMS Bulk Protection Tool and FCI, can be used to create file classification and encryption based on pre-defined rules

Discover and classify: You can configure the FCI automatic classification to automatically classify files that contain sensitive information such as keywords (e.g.: Confidential) or regular expressions (e.g.: \d\d\d-\d\d-\d\d\d\d for a social security combination) on your file server
Protect: Configure the FCI file management tasks to apply RMS protection to all files that are marked as sensitive

Step by step guidance on how to protect sensitive information using FCI and AD RMS is available at: Active Directory Rights Management Services Bulk Protection Tool and File Classification Infrastructure Step-by-Step

File Classification Infrastructure blog posts by Titus labs - Dec. 2009

Nir Ben-Zvi — Thu, 31 Dec 2009 22:06:00 GMT

Additional blog posts by Titus labs that discuss:

Owner based automatic file classification: http://sharepointmetadataandclassification.typepad.com/blog/2009/12/owner-based-classification.html
FCI SharePoint upload script: http://sharepointmetadataandclassification.typepad.com/blog/2009/12/new-file-share-to-sharepoint-migration-utility-from-microsoft.html

Happy new year to everyone

Automatically uploading files from File Server to SharePoint using the File Classification Infrastructure (FCI)

Nir Ben-Zvi — Mon, 14 Dec 2009 22:43:00 GMT

Windows Server 2008 R2 introduced a new File Classification Infrastructure that enables assigning metadata to files and applying file management tasks based on the file metadata (Windows Server 2008 R2 File Classification Infrastructure – Managing data based on business value)

In the past few months, we had a lot of questions from customers on how FCI (www.microsoft.com/fci) can help with scenarios such as:

· Migration from File Servers to SharePoint

· Moving specific files to SharePoint for advanced data management capabilities such as retention …

To help address these scenarios, we have recently published the FCI SharePoint Upload PowerShell script (Upload script on TechNet) that can be used in conjunction with the FCI file management tasks (Customizing File Management Tasks) to upload files to SharePoint (Both SharePoint 2007 and SharePoint 2010 Beta)

Note: you need to enable PowerShell script execution for the script to work

A few points to note about the FCI SharePoint Upload PowerShell script

· When uploading a file to SharePoint, you can choose whether to leave a link on the file server, delete the file or just leave a copy of the file

· The script supports transferring FCI metadata to SharePoint for all file formats as long as the same properties (columns) with a matching name (case sensitive) and type are defined on the SharePoint site

· Files uploaded to SharePoint will get assigned the access rights of the SharePoint library they are uploaded to

An important aspect of the integration with FCI is that it allows you to target which files should be uploaded to the SharePoint site and which should remain on the file server. You can target files based on their creation date, last modified date, extension and of course classification properties.

Targeting which files to upload is a good practice (that will probably save you a lot of time) – for example: for a specific project you probably want to upload only the up to date information and avoid cluttering the SharePoint site.

The example below shows a simple upload scenarios. There are additional scenarios and capabilities that SharePoint 2010 lights up and will be discussed in future blogs

Example: Automatic upload of files into a SharePoint library

Let’s take an example where we would like to upload all files pertaining to the Apollo project from a collection of file shares to a SharePoint document library called “Project Apollo” and leave a links so that users can reach the uploaded files when they browse to the file server.

For this example, we will assume the files that we want to upload are tagged as “Project=Apollo” but this is optional, you could also choose to just upload all files

Some more details before we lay down the steps:

· File server shares are in: d:\projects

· Address of the SharePoint site is http://contoso

· Document library on that site called “Project Apollo”.

· User “contoso\johndoe”, with a password of “up@load”, has the rights to contribute to the site. (this is the user that the script will use when uploading the files)

Using the FCI snap-in (Customizing File Management Tasks)

1. In the right pane, click on “File Management Tasks”.

2. Click on “Create File Management Task…”

This will bring up the “Create File Management Task” dialog.

3. In the “General” tab, fill in the fields in the dialog as follows:

a. Task name example: “SharePoint Upload example”

b. Scope example: “d:\projects”

4. In the “Action” tab select “Type: Custom”

5. Fill in the rest of the fields as follows:

a. Executable: “C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe”

b. Arguments: -noninteractive -file c:\scripts\FciSharePointUpload.ps1 -file “[Source File Path]” -url “http://contoso” -libPath "Project Apollo" –sourceAction url -user contoso\johndoe -password up@load

c. Run the command as: Local System

6. In the “Condition” Tab add a condition such as “Project Equal Apollo”

7. In the “Schedule” tab add a schedule on which you would like to perform the operation

Once you configured the task, you can also choose to “Run task now” and observer the results

What to do when your system misbehave

chfok — Wed, 09 Dec 2009 21:29:00 GMT

Imagine one day you got to your desktop and realized that some programs on your machine started crashing; or you’ve noticed some degradation in your machine’s performance. You’re not so sure when this started happening or what caused it…what should you do?

Action Center is your friend

By now, you’ve probably already figured out that the Action Center is the hub in Windows 7 for finding and resolving issues related to your PC. As a first step, you should always check if there are any problem reports or solutions already available. Action Center can be accessed from the notification area on the task bar:

In addition, Action Center provides two additional options – Troubleshooting and Recovery. The first option provides the tools for diagnosing various computer problems, ranging from Aero disabled to program compatibility issues, and guide you through additional steps such as Remote Assistant if the problem could not be resolved. The second option attempts to fix the issue by restoring the machine to a previous good state. In this blog we’ll focus on the second option – Recovery.

System Restore

In the Recovery control panel, the most prominent option is System Restore. If you’re not familiar with System Restore from previous versions of Windows, it is a tool that would help you roll the state of your operating system back to an earlier point in time, known as a restore point, without affecting your personal files such as documents and pictures. There are several reasons why we recommend System Restore as the first step as oppose to other recovery options. First, it’s the least intrusive option since it rolls back only your system and application files and leaves your personal files intact. Second, if System Restore is initiated from the running OS, an “undo” restore point is created so the process can be reversed. Last but not least, the process is fast and typically takes only a few minutes to complete.

You can access System Restore even if your computer is unable to boot into the operating system. When that occurs, start your computer and hold down F8 to access the recovery option menu where System Restore will be one of the options available. However one thing you should keep in mind is that System Restore initiated from a non-bootable OS will not have the option to undo the operation, though you may be able to attempt to restore to other restore points.

What’s new in System Restore

The System Restore wizard is intuitive and look and feel largely like it was in the previous versions of Windows. In a nutshell, all you need to do is pick a point in time that you would like to roll back to, and then your computer will be restarted to complete the operation.

Even though the experience is familiar, this doesn’t mean there’s no improvement done on System Restore. In fact, here are a few great additions that would help make your system recovery experience even easier in Windows 7:

· View affected programs

Ever wonder what programs or drivers might be affected by performing a restore? Now you don’t need to guess. When you select a restore point, you can see the list of programs, drivers or Windows Updates that you might gain or lose by rolling back in time. You can view this info by selecting the “Scan for affected programs” option in the wizard.

· Using a system image backup as restore point

There’s always a trade-off between how far back in time one can restore to, and the amount of space required on the OS to save older copies of your data for this recovery purpose. Now here’s a new solution. In Windows 7, system images you are already creating as part of your backup to a hard disk can also be used for the purpose of System Restore. Essentially, instead of re-imaging the entire disk, System Restore will extract just the system files from the image and use it like any other restore points. This allows you to potentially roll back to a much earlier point in time, and also an option to perform a less intrusive recovery prior to completely re-imaging your machine. These system image-restore points, if available, can be found in the System Restore UI by selecting “Show more restore points”, with the restore point type indicating that it’s from a backup.

Adjusting System Protection settings in Windows 7 (advanced)

System Restore, Previous Versions and system image backups’ versioning all use the same underlying technology – System Protection (aka Volume Shadow Copies). Here we’ll show you how to adjust settings for these features using the updated System Protection page in Windows 7.

At a high level, System Protection is used to keep track of changes to your personal and system files across an entire drive, such that you can restore them to an earlier state by discarding these changes (via Previous Version and System Restore respectively). The checkpoints that you can roll back to, call restore points, are created during application and driver installations, Windows Update, and at regular intervals if one has not been created in the last 7 days. They can also be created on demand from the System Protection page.

System Protection is turned on by default for the OS drive, but can also be turned on for any additional drives formatted NTFS and greater than 1GB. By default, protection is set to maximum by providing both Previous Version and System Restore capabilities. In Windows 7, however, there’s a new System Protection option “Only restore previous versions of files”. As mentioned earlier, System Restore is designed to bring your system back to an earlier state without affecting your personal files. However, if you are a programmer or system administrator who store programs or scripts on your computer, these files might be undesirably rolled back by System Restore due to their executable nature. By storing these files in a separate data drive and selecting this option, you’ll be able to enjoy Previous Versions protections of such files, but at the same time not having to worry about these files being accidentally rolled back by System Restore. You should not select this option for the OS drive.

For drives 64GB or smaller, a maximum of 3% drive space will be allocated by default for System Protection. For drives greater than 64GB, 5% will be allocated, with a cap at 10GB. You can adjust these settings by selecting the appropriate drive and changing the maximum usage slider. As space runs out, older restore points will be deleted to make room for new ones. Therefore a larger setting will allow more space to be used for storing restore points, which means the ability to restore further back in time. However this also means that less space will be available for other storage purposes on that drive, so it is a tradeoff that needs to be taken into consideration. Similarly, Windows by default allocates a maximum of 30% of disk space on the backup target drive for storing older versions of the system images. This value can be adjusted by using this same control.

The System Protection page also provides the ability to delete all restore points. Note that selecting this option will delete ALL data stored under the system protection storage area, including restore points for System Restore and Previous Versions; earlier versions of system image backups, or any shadow copies created by 3^rd party applications. New restore points will continue to be created unless you turn off system protection entirely. Since System Protection storage has an automatic space management policy, and it can never grow beyond the allocated maximum value, we do not encourage using this functionality if your only motivation is to conserve space.

In the next post, we’ll continue to discuss the rest of the system recovery options on the Recovery control panel, focusing on recovery from a system image.

- Windows Backup team

Volume Shadow Copy (VSS) and Automated System Recovery (ASR) doc updates

dinesh — Thu, 26 Nov 2009 22:28:00 GMT

This is a summary of recent additions and revisions to the VSS and ASR documentation on MSDN and TechNet.

1. The following new API elements have been documented:

a. IVssComponentEx2 interface (http://msdn.microsoft.com/en-us/library/dd405576.aspx)

b. IVssCreateExpressWriterMetadata interface (http://msdn.microsoft.com/en-us/library/dd765211.aspx)

c. IVssExpressWriter interface (http://msdn.microsoft.com/en-us/library/dd405596.aspx)

d. CVssWriterEx2 class (http://msdn.microsoft.com/en-us/library/dd405545.aspx)

e. VSS_RECOVERY_OPTIONS enumeration (http://msdn.microsoft.com/en-us/library/dd433620.aspx)

f. CreateVssExpressWriter function (http://msdn.microsoft.com/en-us/library/dd405544.aspx)

g. IVssHardwareSnapshotProviderEx::ResyncLuns method (http://msdn.microsoft.com/en-us/library/bb427429.aspx)

2. A new topic, "Using Tracing Tools with VSS" (http://msdn.microsoft.com/en-us/library/dd765233.aspx), was added.

3. The Remarks sections for the following methods have been completely rewritten:

a. IVssBackupComponents::EnableWriterClasses (http://msdn.microsoft.com/en-us/library/aa382660.aspx)

b. IVssBackupComponents::DisableWriterClasses (http://msdn.microsoft.com/en-us/library/aa382655.aspx)

c. IVssBackupComponents::DisableWriterInstances (http://msdn.microsoft.com/en-us/library/aa382657.aspx)

4. The in-box VSS writer lists have been expanded. These will eventually be consolidated into a single master list.

a. New In-Box VSS Writers for Windows Server 2008 R2 and Windows 7 (http://msdn.microsoft.com/en-us/library/ee517248.aspx)

b. New In-Box VSS Writers for Windows Server 2008 and Windows Vista SP1 (http://msdn.microsoft.com/en-us/library/ee517247.aspx)

c. In-Box VSS Writers (http://msdn.microsoft.com/en-us/library/bb968827.aspx)

5. The MaxShadowCopies and MinDiffAreaFileSize registry values have been added to "Registry Keys and Values for Backup and Restore" (http://msdn.microsoft.com/en-us/library/bb891959.aspx)

6. The sequence of requester and writer actions in "Overview of the Backup Discovery Phase" (http://msdn.microsoft.com/en-us/library/aa384593.aspx) has been revised.

7. The Events and Errors troubleshooting documentation for VSS (also called the VSS Health Model) (http://technet.microsoft.com/en-us/library/cc734243.aspx) has been updated for Windows Server 2008 R2.

8. The VShadow command has new command-line options for reverting to a previous shadow copy and for resynchronizing LUNs: http://msdn.microsoft.com/en-us/library/bb530725.aspx

9. The registry value information in "Security Considerations for Writers" (http://msdn.microsoft.com/en-us/library/aa384605.aspx) and "Security Considerations for Requesters" (http://msdn.microsoft.com/en-us/library/aa384604.aspx) has been updated.

10. A code example was added to "Using VSS Automated System Recovery for Disaster Recovery" (http://msdn.microsoft.com/en-us/library/aa384630.aspx)

- Diane

Windows 7 Disk Defragmenter User Interface Overview

Jose Barreto - MSFT — Wed, 25 Nov 2009 08:53:00 GMT

Introduction

The Vista and Windows Server 2008 Defragmentation UI was limited to essential information and controls.   Based on user feedback it was found that the UI was too minimal.   In Win7 the UI intends to make the defrag operations more transparent and match the task demands of the user.

List of features provided through Windows 7 Disk Defragmenter UI

1.       Analyze and defragment each volume to improve disk performance

2.       Monitor defragmentation progress and state for each volume

3.       Schedule a job for regular defragmentation

In Windows, defrag is scheduled on an entire volume. The volume can be formatted with the standard windows file systems: NTFS, FAT and FAT32. The volume can be a basic partition or any of the dynamic types: Simple, Stripped, Mirror, RAID-5. The type of disks that host a volume have an impact on automatic scheduling of defrag on the volume. Besides normal rotating hard drives, interesting types of disks in the context of defrag are: Expandable Virtual Hard Disks (VHD) and Solid State Drives (SSD). Defrag cannot be automatically scheduled on volumes hosted by SSD disks and running defrag on volumes hosted by SSD disks is not recommended.

This article explains the different parts of the defrag UI and presents in details the criteria used to determine if the disk hosting the volume is suitable for defrag.

Disk Defragmenter Main UI

This is the primary dialog (Figure 1) that comes up when the defrag UI is started. The UI can be started from the start menu shortcut (All Programs->Accessories->System Tools), from the control panel (Control Panel->System and Security->Administrative Tools-> Defragment Your Harddrive) or from the “Tools” tab in the volume’s properties dialog.

Figure 1

Schedule

The “Schedule” section in this dialog (figure 1) displays volumes that are scheduled for automatic defragmentation. It is recommended to have the schedule turn on. The new Windows 7 Disk Defragmenter will exclude the volumes that are not recommended to run defrag regularly. [ Note even if the system only has SSD disks and expandable VHDs, the schedule would still be turned on; to verify if the volumes are scheduled for defrag click Configure Schedule.]

Current Status

The list shows all the volumes that can be defragmented. This includes volumes on the solid state drives. Note that volumes on SSDs and dynamic VHDs can be defragmented but are not scheduled to run regularly; this gives users the options to defrag SSD volumes if they feel the need to do so.

To analyze disk or Defragment Disk: select the volume from the list, and then click on Analyze disk or Defragment disk.

Configure Schedule

This dialog (figure 2) allows modification of the defrag schedule. Click on Select disks to see the volumes that are included in the defrag schedule.

Figure 2

Select Disks/Volumes

Select or unselect the Checkboxes to include or exclude the volumes for regular defragmentation respectively. Selecting “Automatically defragment new disks” will automatically add new volumes to the defrag schedule

Figure 3

The screenshot below (figure 4) shows there are no volumes scheduled for defragmenter. No volumes are listed because all the volumes in the system are not suitable for regular defrag (because they could all be on SSDs or expandable VHDs). However, the user can go to the main defrag UI to defragment the volume manually.

Figure 4

What volumes are excluded from defrag schedule?

As mentioned earlier, regular defragmentation of volumes on certain disk types like SSD disks and expandable VHDs are not recommended. Defrag UI excludes volumes from being scheduled based on the following:

·         Volumes on disks whose driver reports “no seek penalty”. [more details …]

·         Volumes on ATA/SATA disks that report a nominal media rotation rate of 1. [more details …]

·         Volumes on critical disks with high very random read disc rates. [more details …]

·         Volumes on expandable VHDs.

The high level overview of the detection algorithm as follow:

                If (no_seek_penalty) {

                                exclude from schedule

} else if (is_expandable_VHD || is_differential _VHD) {

                                exclude from schedule

} else if (is_ATA_nominal_disc_rate) {

                                exclude from schedule

} else if (is this boot disk) && (high_random_read_rate) {

                                exclude from schedule

}else{

                                include in the schedule

}

Note that volumes on any VHD which in turn is hosted on a SSD is also excluded from defrag.

How does disk defragmenter get the “no seek penalty” from device?

Disk port driver needs to send a valid response to IOCTL_STORAGE_QUERY_PROPERTY call. Disk Defragmenter issues IOCTL_STORAGE_QUERY_PROPERTY request with QueryType = PropertyStandardQuery and PropertyId = StorageDeviceSeekPenaltyProperty.

A valid response involves populating a DEVICE_SEEK_PENALTY_DESCRIPTOR structure with accurate version and size data. If the IncursSeekPenalty member is set to false, the disk is considered a SSD.

How does disk defragmenter get the “nominal media rotation rate” from device?

If the port driver does not return valid data for StorageDeviceSeekPenaltyProperty, Disk Defragmenter looks at the result of directly querying the device through the ATA IDENTIFY DEVICE command. Defragmenter issues IOCTL_ATA_PASS_THROUGH request and checks IDENTIFY_DEVICE_DATA structure. If the NomimalMediaRotationRate is set to 1, this disk is considered a SSD.

The latest SSDs will respond to the command by setting word 217 (which is used for reporting the nominal media rotation rate to 1). The word 217 was introduced in 2007 in the ATA8-ACS specification.

How does disk defragmenter get the “random read disc score” from the device?

Disk Defragmenter uses the Windows System Assessment Tool (WinSAT) to evaluate the performance of the device. The performance threshold was determined by some I/O heuristics through WinSAT to best distinguish SSD from rotating hard disks.

------

Post by Eva Liu

Managing backup disk space

chfok — Mon, 23 Nov 2009 06:50:00 GMT

If you’re reading this post it’s likely that you’ve already set up Windows Backup on your Windows 7 PC and running it on a schedule to protect your data and system. However as storage size is limited, your backup disk will run out of space at some point. Don’t sweat it. Windows 7 makes it easy to delete older file backups and manage the number of image backup versions.

When to manage backup disk space

You can manage backup disk space at any time using the “manage space” option from the Backup and Restore control panel. , but you’ll also get a notification from Action Center when backup cleanup becomes necessary:

When you click on this message, you will see the space management window opens which has 3 sections:

· Space usage summary: Shows the total amount of space on the backup target, and a breakdown of how the space is currently utilized by file backup, system image, and any other files you may have.

· Data file backup: Provides the option to delete data file backups on target to free up space.

· System image: Lets you configure the number of system images stored on the target.

We will discuss these sections in detail below.

Space usage summary

This section is fairly straight forward. Its goal is to help you determine what is occupying your backup disk space and decide what to delete. Note that if you have backups from another computer present on the backup disk, those will also be shown under “Other Files”. If you decide to delete the non-backup files on the target, you can conveniently browse the target from the “Browse” link below the icon on the left.

Data file backup

As a recap from the previous blog post Protect your files and PC with Windows 7 Backup<link> - when you configure Windows Backup, the first backup will be a full backup of all the selected backup contents. Subsequent backups will be incremental to the incremental in nature, containing only new or changed files since the previous backup. Windows Backup will, however, automatically create full backups from time to time. This is such that you will have the option to delete older backups that may contain deleted or obsolete files that are no longer useful, without losing the backup of important files that are still present on your machine. The frequency of creating new, full backups depends on several factors, such as the disk space consumed by deleted or older versions files.

When you select the “View backups…” option, you’re presented with a dialog that shows all the backup of your current computer on the target. The granularity for deletion is a backup period, which represents a full backup together with its associated incremental backups. Whenever you run out of backup disk space, you should always start deleting from the oldest backup period to minimize potential data loss. If there is only one backup period available, you should really think twice before deleting it since you will lose your entire backup. You may want to copy that backup to another location, or try to delete other non-backup files on the target to free up space instead. If you do choose to go ahead with the removal, Windows Backup will offer to kick off a new, full backup to ensure you maintain up-to-date protection of your data.

System image

Your options for managing system image depend on whether you’re currently creating system images on a scheduled basis.

1. Creating system image on schedule

When you set up Windows Backup to create system images on a regular basis, it’ll automatically manage the amount of space used for storing the backup versions. As explained in the system image backup post <link>, older system images are stored in the shadow copy storage area on the backup disk. By default, it’ll set this shadow copy storage area to use a maximum of 30% of the backup disk space. Windows Backup will move older backups to this storage area as new ones are created, until the maximum allocated space is filled. At that point, the oldest backup will be deleted, followed by the second oldest and so on, until enough space is freed up to allow the latest image to be created.

This default behavior is shown above as the pre-selected option “Let Windows manage the space used for backup history”. In this example, 30% of the backup disk space amounts to 81.13GB. The reason that not the entire disk space is utilized for storing older system images is so that there will be space left for storing your file backup as well. If you do wish to adjust the size of the shadow copy storage area (though not recommended), you can accomplish this by following these steps:

i. Go to Start->Computer->Properties->System Protection.

ii. Choose your backup target, click “Configure”

iii. Under “Disk Space Usage”, adjust the slider to the desired setting

When you adjust this setting, keep in mind that increasing the shadow copy storage area size comes at the expense of the space that can be used for other purpose, most importantly your file backup. Another thing is that you should not turn on System Protection for your backup disk, as this will enable the shadow copy storage area to be used for other purposes which will accelerate the deletion of your older backups. We’ll discuss the System Protection page in details in a later post when we look at some of the system recovery improvements in Windows 7.

If you only care about having the latest system image available and would like to conserve disk space, you can select the second option in the dialog shown above. Choosing this option means that Windows will always keep one (the latest) system image only for your computer. Whenever a new system image is created, the older one will be deleted.

Note that these options are available only if you’re creating scheduled system image to a hard disk. If you’re backing up to a network share, these options are not available since network share does not support storing older system images. Older system image of the same computer will always be deleted when a new one is created.

2. System images are not created on a schedule

If you’ve previously created system images on the target but are no longer creating any new images on a schedule, you can choose to retain the latest image and delete the rest, or delete all system images on the target. We recommend that you always keep at least one system image around for safekeeping.

With hardware storage price getting much lower these days, we recommend investing in a larger backup disk to protect your important data. However even when you do run out of space, Windows Backup makes it easy to manage older backups to ensure that data protection is not disrupted. In the next post we’ll cover the system recovery options in W7.

-- Windows Backup team