<This is the third post in a blog series on backup and recovery offerings in Windows 7. >
Many of us have experienced this panic moment when we realized some important files are missing or are accidentally modified, but Windows users who’ve configured Windows Backup (or even those who haven’t) don’t need to sweat, because their data is in good hands. In this post we’ll talk about how to recover a file from the local hard disk and from a backup. As an advanced topic we’ll also discuss how to extract files from a system image.
Default protection
We cannot emphasize enough the importance of having a backup of your data on an external storage location (e.g. external hard disks, network location) as that’s the only way to safeguard your important data against hardware failure on your PC. That being said, by default Windows will automatically create copies of your data on your OS drive to help ease other data loss situations such as accidental deletion and modifications. These copies of files are created on a best effort basis and may be short lived, so you should never rely solely on them as a replacement for a backup. Nonetheless they could be extremely useful if you’re unable to access a backup for any reason, such as when you’re on the go.
(Note that these automatic copies, also known as system protection, are turned off by default on non-OS drive. If you’re interested in learning more about this feature, remember to read our next post as we’ll discuss them in detail in conjuncture with the System Restore feature.)
To access an older version of a file, simply right click on the file and select “Restore previous versions”, where you will see all the available versions of the file. You can then choose to restore the file, or as a safer option copy the file to another location. Alternatively if the file has been deleted, you can select the same option on the folder that originally contained the deleted file, open the version that contains the deleted file and do the restore.
Restoring files from a backup on the same computer
If you’ve set up Windows Backup on your computer, you can also access the files in the backup through the same dialog above; except you’ll see from the location column that the file is in “Backup” rather than in a “Restore point”. This experience is optimized for quick restore of a single file, and does not work for restoring an entire folder or if you need to restore selected files from various folders. In these cases you can launch the file recovery wizard from the Backup and Restore control panel to access the complete list of file recovery options.
When you click on “Restore my files” from the control panel, you’ll be able to restore all the files that you have read access on that were backed up to the current backup target. If you want to restore other users’ files, choose the “Restore all users’ files” option instead, which would require you to have administrative privilege. This option would also allow you to restore files to locations that may require administrative access. To select files or folders for restore, you can either browse your backup content, or you can search for the file\folder by its name or using wild card searches. By default, the restore wizard will always show you the latest versions of all your files. If you need an older version of a file, select “Choose a different date”.
After selecting which file/folder to restore, you can then select where to restore the files to. To avoid potentially dealing with file name conflicts or accidentally overwriting your data, we would recommend restoring to an alternate location and then copying the files back only after verifying that those are truly the ones you want. Also, if you’re restoring applications or system files, you will not be able to directly restore them to original location since doing so might break existing applications (and in most cases simply replacing such files would not help repairing an application or system state. To revert your system or application to an earlier point in time, System Restore should be used. See our next post to learn more about this feature). Attempt to restore these files to original location will result in those files being skipped and reported in the log file. If you do need to restore such files, restore them to a different folder and then copy them to the desired location.
After the restore is completed, you can launch explorer from the restore wizard to view the restored files.
Restoring files to a new OS or after recovering from a system image
If you have reimaged your computer to an earlier point in time, or installed a new operating system, the recovered or new OS will not have record of the latest backups that you’ve made. In this case, you can use the “Select another backup to restore files from” option to specify where the backup is located. The wizard will automatically look for available backups on all devices currently connected to the computer, and you can also specify a network location to restore from. Once you’ve selected the backup, the rest of the steps to restore the files remain the same.
When restoring files from a different OS, it’s highly recommended that you restore the files to an alternative location since the users and folder structures on the new computer might have. Restoring to original location means that the restore wizard will restore your files by recreating the old folder paths, which might make it much harder locate the restored files afterwards.
Restoring backups from previous versions of Windows
You can restore backup made from Windows Vista following the same steps above as restoring a Windows 7 backup to another computer. If you’ve created a backup using the NTBackup utility in XP, at the time of this blog post you can download a free tool here to restore the .BKF files on Windows Vista. The ability to restore .BKF files on Windows 7 will be supported in the near future and more details will be provided at that time.
Extracting files from a system image (advanced)
As discussed in previous blog posts, system image is a snapshot of your computer created at the block level, with the latest image stored in a Virtual Hard Disk format (.vhd file). Even though a system image is meant to be used only for rebuilding your computer, due to its storage format it’s possible to mount the image as a virtual volume on your computer to browse or extract its content. This has been made a lot easier in Windows 7 due to the new set of VHD features available from the disk management console.
To access this option, simply go to Computer Management (Right click Computer->manage) and select the Disk Management option. Right click and you’ll get an option to attach a VHD. By selecting the system image VHD and mounting it, you will now be able to browse the latest system image of your computer. Make sure you do not modify the content of this virtual drive or you’ll risk losing important backed up data. Also, you should detach the VHD (also from the management console) after you are done browsing otherwise the next backup cannot be run.
Sometimes bad things do happen to good people, but hopefully this post has provided you with all the information you need to know about recovering your valuable data. Coming up in the next few posts, we’ll cover how to manage space for backup, and then move onto system recovery related topics.
-- Windows Backup team
<This is the second post in a blog series on backup and recovery offerings in Windows 7. >
In the last post, we’ve briefly mentioned that Windows 7 offers both the ability to backup individual folders and to create a system image of your computer, and how they could be used to protect your personal and system files. In this post we’ll focus on system image backup to give you an in depth view on how the technology works, and how the configuration of your OS may affect the image and the implications during recovery.
What is a system image and how does it work?
Just as a brief recap, a system image is in essence a snapshot of an entire drive(s). The backup is done in block level (as opposed to file level) increments and includes all user and system files, configuration data and applications that are present on the drive, plus information regarding disk layout and boot entries. The image can be used to recover a working Windows if your hard disk ever fails, or if you simply want to reimage your OS to an earlier point in time.
During the first backup, the backup engine scans the source drive and copies only blocks that contain data into a .vhd file stored on the target, creating a compact view of the source drive. The next time a system image is created, only new and changed data is written to the .vhd file, and old data on the same block is moved out of the VHD and into the shadow copy storage area. Volume Shadow Copy Service is used to compute the changed data between backups, as well as to handle the process of moving the old data out to the shadow copy area on the target. This approach makes the backup fast (since only changed blocks are backed up) and efficient (since data is stored in a compact manner). When restoring the image, blocks will be restored to their original locations on the source disk. If you want to restore from an older backup, the engine reads from the shadow copy area and restores the appropriate blocks.
Creating a system image
As mentioned in the previous posts, when configuring Windows Backup, a system image is automatically included in the scheduled backup if the backup target is formatted with NTFS file system and has sufficient space. This system image contains only the critical drives that are required for Windows to operate. Examples of critical drives include system volume, boot (OS) volume, and the volume where the Windows Recovery Environment is installed (typically the same as boot volume on a default installation of Windows 7). As seen from the example below, the System Reserved drive, which is the system volume, and the C: drive that represents the boot volume are both included. The G: drive on the other hand, which is purely a data drive, is not included. To back up any data from G:\, you can select the drive or its folders from the tree view above to create a file-based backup.
Using the Windows Backup wizard above is the simplest way to protect your data and system, since the backups will be created automatically on a schedule. However if you want to create an advanced system image that also includes additional data drives on top of the critical drives, or if you want to save a custom system image to an alternate location (e.g. to a set of DVDs that you can easily carry with you on a trip), you can create such images on an ad hoc basis using the “Create a system image” task link on the Backup and Restore control panel.
The steps to create an advanced system image are very similar to creating the regular scheduled backup. First, select where you would like to save the image, which can be the same as or different from the target for the regular scheduled system image backup. Then you can select which drives should be included in the image. Some drives are selected by default since they are required for Windows to run and must be included in the image. However you are free to select any additional drives to include.
After a final confirmation of the source and the target, you are good to go.
Command-line options
Wbadmin.exe is the command line utility for creating system images and supports all functionalities available from the wizard and more. Using wbadmin and task scheduler together you can create advanced system image on a scheduled basis. For more information regarding the wbadmin subcommands, type “wbadmin help” from a command prompt or visit MSDN (note that some subcommands on MSDN are only available for Windows Server 2008 backup).
There are no differences in the system image created using a wizard or command line tool, and you will be able to restore the images using the same procedures (more on system image recovery in a later post). However, the configurations of your source and target disks do have implications on the ability to backup and restore the image. We’ll explore this some more in the next section.
Considerations while creating a system image
Since system image is a critical feature to ensure availability of your system and data after a disaster, it is important to understand how some of the advanced configuration on your system may affect your options during restore.
1. Choosing the backup target
System image is supported on internal\external disks, optical removable media, and network locations (Business edition or above). Aside from the usual tradeoffs when picking a storage location such as performance and reliability, here are some additional recommendations to consider for picking a system image backup target:
- Do not store the image on the same physical disk as the OS – If the hard disk ever fails, you’ll lose both your OS and the backup.
- Do not store the image on a dynamic disk – a system image stored on a dynamic disk will provide limited functionality during restore. Specifically, the restore will be supported only if the partitioning on the source disk has not been changed. Therefore if your hard disk failed and is replaced with new hardware, you will not be able to restore the image.
- Support for multiple images – As mentioned above, when a new system image is created, older data will be moved to the shadow copy storage area if it’s available. NTFS formatted internal or external hard disks supports Volume Shadow Copy, hence they support storing multiple versions of backup image. We recommend that the size of the target disk should be at least 1.5 times of the size of the source drive(s) to allow enough space for storing older versions of backup. Network share and optical media, on the other hand, do not support shadow copy. Therefore only one system image per computer can be stored at a time (as the .vhd file). Any newer backup created will replace the older backup.
- Use a dedicated backup disk - Older backup that are stored in the shadow copy area on the target may be deleted as space runs out. The amount of churn that happens on the target is one factor that contributes to this deletion. Therefore to ensure the older backups can be kept for a longer period of time, try not use the backup disk for other purpose.
More information regarding differences between backup on hard disk, optical media and network locations can be found in Help and Support.
2. Configuration on the source disk(s)
When a system image is created, it captures the data of (at least) all critical drives on the source disks, and also information regarding the state of all disks and partitions that are present in the system in case partitions need to be recreated during restore. Therefore the layout of the disks at the time of backup will have implications on what is included in the backup and also the options available for restore.
- Disk layout when dual booting Windows 7 with an earlier operating system – Depending on how the two OS are installed, the earlier operating system may be residing on a drive considered critical for Windows 7 and therefore will be mandatorily included in the system image for Windows 7. For example, if the computer originally has Windows Vista installed on the (C:) drive which serves as both the boot and system volume, then Windows 7 will automatically make use of (C:) as its system volume as well. This makes (C:) a critical volume for Windows 7 and must be backed up. However this also means that the all data associated to Windows Vista will be included. To avoid including an earlier OS in the Windows 7 image, you will need to create a separate system partition to be shared by both OS. We’ll be posting an article soon on how this can be done.
- Create a new image after a disk layout change – During system image recovery, the UI will check if the current layout on the disk matches the layout information captured in the backup. If it matches, then it will allow restoring only the data in the critical partitions without formatting and recreating partitions on the entire disk. However if the layout has been changed, then it will need to recreate the partitions using information from the backup, which means formatting the entire disk and potentially deleting any non-critical partitions and their data. Therefore it’s recommended that if you ever change the disk layout on your computer, you should create a new system image to allow for a more flexible recovery experience.
3. Multiple machines environment or dual boot machine
- Use different computer names - System images are identified by computer name, and are stored in separate folders with the corresponding computer name on the backup target. If you have two computers with the same name and you use the same backup target for both, the system image of the second computer will overwrite the first one (If the backup target supports shadow copy, this means the older backup will be moved to the shadow copy storage area and may be lost as space runs out). This is especially important if you have created a system image of an earlier OS (e.g. Windows Vista) and then upgraded to Windows 7 using the same computer name, the Vista image may be lost.
- Do not access the backup disk with Pre-Windows Vista OS – Pre-Windows Vista OSes (e.g. Windows XP) do not support the current shadow copy mechanism, and will cause shadow copies to be deleted if it attempts to access a target with shadow copies present. In this case, all older backup (except the latest image stored in the .vhd file) will be lost.
If you have any questions regarding your specific configurations, do not hesitate to leave a comment or visit our forum for support.
-- Windows Backup team
My previous blog post explained how to configure the DFS Replication management pack. The backlog monitoring feature of the management pack is disabled by default and requires some additional configuration. This blog post explains how to enable backlog monitoring using the DFS Replication management pack and also describes how to override some of the default configuration settings in the management pack.
Configure the DFS Replication monitoring profile
In Operations Manager 2007, Run As Profiles and Run As Accounts are used to select users with privileges that are needed to run rules, tasks, and monitors. The backlog monitoring discovery script included in this management pack queries the DFS Replication WMI provider on all monitored computers. To do this, it needs to run in the context of a privileged monitoring account. This management pack includes a privileged monitoring profile called DFS Replication Monitoring Account. You need to add a Run As account to this profile to monitor the replication backlogs.
The first step is to create a Run As account that has the requisite privileges to connect to the DFS Replication WMI provider on all monitored computers. The WMI provider does not allow non-administrative access and therefore, you need to ensure that the Run As account has at the very least local administrator privileges for each of the monitored computers.
-
To create a Run As account on System Center Operations Manager 2007 R2, perform the following steps:
-
Open the Operations console with an account that is a member of the Operations Manager 2007 Administrators role.
-
Select the Administration view.
-
In the Administration view navigation pane, right-click Accounts, and then select Create Run As Account.
-
In the Create Run As Account Wizard, on the Introduction page, click Next.
-
On the General Properties page, do the following:
-
Select Windows in the Run As Account type list.
-
Type a display name in the Display Name text box. Choose a descriptive name such as DFS Replication Monitoring Users.
-
You can also type a description in the Description text box. Adding a description helps other users know why you set up this account and the privileges associated with it.
-
Click Next.
-
On the Credentials page, type the user name and password of the account with administrative privileges that you want to use, select the appropriate domain, and then click Create.
-
On the Distribution Security page, select More secure – I want to manually select the computers to which the credentials will be distributed, and then click Create.
After the Run As account has been created, right click on the Run As Account in the Accounts pane and select its properties. In the Run As Account Properties – DFS Replication Monitoring Users dialog, select the computers to which these credentials are to be distributed.
The next step is to associate this Run As account with the privileged monitoring profile included in the DFS Replication management pack. This configuration step ensures that the replication backlog discovery script has the required privileges to connect to the WMI provider for DFS Replication on monitored computers and retrieve replication backlogs. To add the Run As account you just created to the DFS Replication Monitoring Account Run As Profile on System Center Operations Manager 2007 R2, perform the following steps:
-
Open the Operations console with an account that is a member of the Operations Manager 2007 Administrators role.
-
Select the Administration view.
-
In the Administration view navigation pane, click the Profiles container.
-
In the list of available Run As profiles, right click DFS Replication Monitoring Account, and then select Properties.
-
In the Run As Profile Wizard, on the Introduction page, click Next.
-
Click Next on the General Properties page.
-
On the Run As Accounts page, click Add.
-
In the Add a Run As Account dialog box, select the Run As account you created from the list of available Run As accounts, for example DFS Replication Monitoring Users.
-
Select the All targeted objects option to manage all monitored computers by using this Run As profile, and then click OK.
-
Click Save to update the Run As Profile.
Enable backlog monitoring
This feature enables administrators to keep track of replication backlogs on monitored computers. Backlog tracking is performed by a discovery script that queries the WMI provider for the DFS Replication service on monitored computers. This discovery script is disabled by default, thereby disabling the monitoring of replication backlogs by default. This section provides instructions for enabling backlog monitoring. After backlog monitoring is enabled, the Backlog Monitoring dashboard view will be populated with replication backlogs that are retrieved from all monitored computers.
To enable backlog monitoring for the DFS Replication Management Pack, perform the following steps:
-
Log on to the computer with an account that is a member of the Operations Manager Advanced Operator role for the Operations Manager 2007 management group.
-
In the Operations console, click Authoring.
-
In the Authoring pane, expand Management Pack Objects, and then click Object Discoveries.
-
In the Object Discoveries pane, completely expand Replication Connection.
-
In the results pane, select the discovery rule titled Discover replication backlogs on monitored computer.
-
On the Operations Manager toolbar, click Overrides, and then point to Override the Object Discovery. You can choose to override this monitor for objects of a specific type or for all objects within a group. We recommend that you create an override with the scope ‘For all objects of type: Replication Service’.
-
After you choose the objects for which you would like to override the default settings, the Override Properties dialog box opens, which enables you to view the default settings that are configured for the backlog discovery rule.
-
Click to place a check mark in the Override column next to each setting that you want to override. When you complete your changes, click OK.
-
As shown in the following example, to enable the backlog monitoring discovery script, select the parameter Enabled, and then set its Override Setting to TRUE. The example also shows how you can configure the default frequency with which the script is run.
-
You can also configure the frequency with which the backlog monitoring discovery script runs by overriding the value of the Interval in seconds parameter. The script is set to run with a default frequency of 14,400 seconds (4 hours). By changing the value of this parameter, you can configure the backlog discovery script to run at a desired frequency.
-
At the bottom of the Override Properties dialog box, you can select the management pack in which to store the override settings. We recommend that you create a separate management pack to store override and custom configuration settings that are specific to the DFS Replication Management Pack. Remember to name the separate management pack clearly so that you can easily find it and keep a backup of that management pack.
Note: We highly recommend that you store all override settings in a separate management pack. By default, if you do not choose to store settings in a separate management pack, all overrides and custom configuration settings are stored in the Default Management Pack.
-
Click New. The Create a Management Pack Wizard appears. Choose a name for the management pack in which to store the override settings. Optionally, if you have already created a new custom management pack to store the override settings and other customizations for the DFS Replication Management Pack, select the name of that management pack in the Override Properties dialog box.
-
After you finish, click Apply, and then click OK to create the custom override and to enable backlog monitoring for the DFS Replication Management Pack.
Note that it will take some time before the Backlog Monitoring view is populated with replication backlogs that are retrieved from the monitored computers.
Configure how often discovery rules are run
The following table lists the default frequency with which various discovery rules that are included in the DFS Replication Management Pack are run. Evaluate these default settings to determine whether the configured default frequencies are appropriate for your environment. If a configured default frequency is not appropriate for your environment, consider overriding the corresponding discovery rule and tuning the frequency as appropriate.
| Discovery Rule | Default Frequency |
| Discover replication backlogs on monitored computer. | 4 hours |
| Discover replication groups on monitored computer. | 4 hours |
| Discover DFS Replication settings on monitored computer. | 4 hours |
| Discover volumes hosting replicated folders on monitored computer. | 4 hours |
| Discover replicated folders on monitored computer. | 4 hours |
To tune the configured default frequency for discovery rules that are included in the DFS Replication Management Pack, perform the following steps:
- In the Operations console, click Authoring.
- In the Authoring pane, expand Management Pack Objects, and then click Object Discoveries.
- In the Object Discoveries pane, completely expand any of the following discovered types:
- Replicated Folder
- Replication Member
- Replication Service
- Replication Volume
-
Select the discovery rule that you would like to tune.
-
On the Operations Manager toolbar, click Overrides, and then point to Override the Object Discovery. You can choose to override this monitor for objects of a specific type or for all objects within a group.
-
After you choose which group of object types to override, the Override Properties dialog box opens, which enables you to view the default settings that are configured for this object discovery.
-
Click to place a check mark in the Override column next to each setting that you want to override. When you complete your changes, click OK.
-------
Mahesh Unnikrishnan
My previous post explained how to import the DFS Replication management pack using the Operations Manager console. In this post, let’s explore how to set up the management pack and configure it to monitor the health of DFS Replication.
Select computers to be monitored
The first step is to configure Operations Manager to monitor the servers running DFS Replication that make up your replication infrastructure. This can be done using the Administration option in the Operations Manager console. To configure computers to be monitored using Operations Manager, perform the following operations:
- In the Administration pane, click Device Management.
-
Select Windows computers in the wizard.
-
In the following page, select the Automatic computer discovery option.
- Follow the instructions in the wizard and let the discovery scan run.
- Remember to select Agent as the Management Mode in the wizard.
After this, the wizard proceeds to install the agent on all computers you have selected for monitoring.
Once the agent has been installed on all the computers you have selected for monitoring, you should be able to see entries for each of the computers in the list of Agent Managed computers. By the end of this step, Operations Manager will also be done pushing the DFS Replication management pack out to these monitored computers.
At this stage, the management pack is now setup to perform basic monitoring for the DFS Replication service. You will notice that it has discovered that the DFS Replication service is running on the computers you’ve selected for monitoring.
However, for many of the discovery rules to work properly, we need to perform some post-install configuration. For example, you will notice that the management pack does not discover replication group members and settings configured for them. You will also notice that the Backlog Monitoring is empty. These features need some additional configuration settings.
These post installation configuration steps are detailed below.
Configure the management pack
After you import the DFS Replication Management Pack, follow these configuration steps.
1) Enable the Agent Proxy setting on all monitored computers
If you do not enable the Agent Proxy setting on servers running the DFS Replication service, the discovery rule ‘Replication Group Discovery’ fails to run. To enable the Agent Proxy setting on all computers you would like to monitor, perform the following steps:
- Open the Operations console, and then click Administration.
- In the Administration pane, click Agent Managed.
- Double-click the name of a server running the DFS Replication service.
- Click the Security tab.
- Select Allow this agent to act as a proxy and discover managed objects on other computers.
- Repeat these steps for each server running the DFS Replication service that you want to monitor.
2) Allow WMI through the Windows Firewall on all monitored computers
The DFS Replication management pack uses Windows Management Instrumentation (WMI) scripts extensively for its discovery rules. Therefore, it will not work if a monitored computer’s firewall has been configured to disallow WMI connections.
3) Create a new management pack for customizations
Most official management packs (including the DFS Replication Management Pack) are sealed so that you cannot change any of the original settings in the management pack file. However, you can create customizations, such as overrides or new monitoring objects, and save them to a different management pack. By default, Operations Manager 2007 saves all customizations to the Default Management Pack. As a best practice, you should instead create a separate management pack for each sealed management pack that you want to customize. Creating a new management pack for storing overrides has the following advantages:
-
It simplifies the process of exporting customizations that you created in your test and preproduction environments to your production environment. For example, instead of exporting the Default Management Pack that contains customizations from multiple management packs, you can export only the management pack that contains customizations for a single management pack.
-
You can delete the original management pack without first deleting the Default Management Pack. A management pack that contains customizations is dependent on the original management pack. This dependency requires that you delete the management pack with customizations before you can delete the original management pack. If all of your customizations are saved to the Default Management Pack, you must delete the Default Management Pack before you can delete an original management pack.
-
It is easier to track and update customizations to individual management packs.
For more information about management pack customizations and the Default Management Pack, see About Management Packs in Operations Manager 2007 (http://go.microsoft.com/fwlink/?LinkId=108356).
Optional configuration – enable backlog monitoring
The next blog post explains how to perform optional configuration steps for the DFS Replication management pack. This includes enabling the backlog monitoring feature.
-------
Mahesh Unnikrishnan
My previous post explained how to install the DFS Replication management pack on the management server. The next step is to import the management pack using the Operations Manager console. This post explains how to import the management pack.
Before you import the DFS Replication management pack
Upgrades from the beta or other prerelease versions of the DFS Replication Management Pack are not supported. You must delete the beta or prerelease version of the management pack by using the Operations console before you install this version. When you delete a management pack, all the settings and thresholds that are associated with it are removed from Operations Manager 2007. Also, the .mp or .xml file for that management pack is deleted from the hard disk drive of the management server. If you do not have a prerelease or beta version of the DFS Replication management pack, feel free to skip this section.
Perform the following steps to remove an existing beta version of the management pack prior to installing this version:
-
Log on to the computer with an account that is a member of the Operations Manager Administrators role for the Operations Manager 2007 management group.
-
In the Operations console, click Administration.
-
In the Administration pane, click Management Packs.
-
In the Management Packs pane, right-click Microsoft Windows DFS Replication, and then click Delete.
-
Click Yes when you see the confirmation dialog.
After you remove the beta or prerelease version of the management pack, restart the Operations Manager health monitoring service on the monitored computers. This can be done by running the following commands on the monitored computers:
net stop HealthService
net start HealthService
After the health monitoring service has started, you are ready to import the new DFS Replication Management Pack and begin monitoring DFS Replication.
Importing the management pack
There are multiple options when it comes up to importing a management pack. These are explained in detail on TechNet. Broadly speaking, these options are:
- Import directly from the System Center Operations Manager 2007 Catalog by using the Operations console.
- Import from disk (local storage or a network file share) by using the Operations console.
- Use the Operations console to download a management pack from the catalog to import at a later time.
- Use an Internet browser to download a management pack from the catalog to import at a later time.
This blog post explains the second option for importing the management pack. As explained in the previous blog post, the DFS Replication management pack would be installed on the management server in the ‘System Center Management Packs’ folder under the Program Files directory. The detailed steps are given below:
-
Log on to the computer with an account that is a member of the Operations Manager Administrators role for the Operations Manager 2007 management group.
-
In the Operations console, click Administration.
-
The Import Management Packs wizard opens. Click Add, and then click Add from disk.
-
The Select Management Packs to import dialog box appears. If necessary, change to the directory that holds your management pack file. Click one or more management packs to import from that directory, and then click Open.
-
On the Select Management Packs page, the management packs that you selected for import are listed. An icon next to each management pack in the list indicates the status of the selection, as follows:
-
A green check mark indicates that the management pack can be imported. When all management packs in the list display this icon, click Import.
-
A red error icon indicates that the management pack is dependent on one or more management packs that are not in the Import list and are not available in the catalog. To view the missing management packs, click Error in the Status column. To remove the management pack with the error from the Import list, right-click the management pack, and then click Remove.
-
The Import Management Packs page appears and shows the progress for each management pack. Each management pack is downloaded to a temporary directory, imported to Operations Manager, and then deleted from the temporary directory. If there is a problem at any stage of the import process, select the management pack in the list to view the status details. Click Close.
After the management pack has been imported using the Import Management Pack wizard, you will notice that there is a new entry for the DFS Replication Management Pack as shown in the screenshot below.
You’re now ready to configure the management pack and setup customizations suitable for monitoring your environment. This will be covered in the next blog post.
-------
Mahesh Unnikrishnan
Anyone who uses a Windows 7 PC shouldn’t have to worry about losing their files, because Windows 7 provides simple yet flexible backup and recovery solutions that helps protect your system and data. In the coming weeks, we’ll walk you through each step of the process – from configuring backup, restoring your OS, to recovering an entire PC. Stay tuned for the updates!
Special thanks to Sneha Magapu, Neha Agrawal, Vikas Ranjan and Soudamini Sreepada for their contributions to the posts.
Configuring Windows Backup
Have you ever accidently deleted or modified a file and wished you get it back? Or worse, have you ever lost all of the data on your computer because your hard drive failed? Windows 7 aims to help you be well prepared for these situations by making backup easy to discover and simple to use.
Setting up a Backup
Windows reminds you to configure backup
After spending some time personalizing your brand new Windows 7 computer and migrating your data from your old computer using Windows Easy Transfer, it’s a good time to start backing up. Windows will remind you to set up backup through Action Center one week after setting up Windows 7:
1. Choosing a backup location
Setting up backup is as simple as answering 3 questions – where, what and when. By plugging in a suitable external hard drive, the first question is already answered. An AutoPlay dialog will give you the option to use the drive for backup.
By clicking “Use this drive for backup”, you can then proceed to choosing what to backup. Alternatively, you can start configuring backup from the Action Center notification. The configuration starts with the target selection page, which allows you to choose where you save your backups. Windows Backup supports back up to hard disk drive, network share, or CD\DVD.
This page automatically lists all available drives that can be used for backup, and provides recommendation on the best option if more than one is available. You can also add a network location if desired. Note that there are some restrictions on the locations allowed for backup, including locations with size less than 1GB, the drive being the same as the one Windows is installed on, or if the drive is currently locked by BitLocker. If you do not see your backup target listed, click on the help link ‘Guidelines for choosing a backup’ for more information.
There are also pros and cons for choosing different locations for backup. We’ll discuss this in detail later in the post.
2. Choosing what to back up (or let Windows decide for you)
Some users have a hard time deciding what to back up; others would like to be in control. This screen is designed to make it simple for the common case and also allow for customization. By selecting the “Let Windows Choose” option, Windows will back up all libraries (both default and the ones you’ve created yourself) and default Windows folders (AppData, Contacts, Desktop, Downloads, Favorites, Links, Saved Games and Searches) for all user accounts on the computer. If the backup location is formatted with the NTFS file system and has sufficient space, a system image will also be included. This system image is in essence a snapshot of the drives required for Windows to run, which includes your programs, data, and settings. It can be used to quickly recover Windows to a last backed up state, particularly if your hard disk ever stops working. If you’re not sure whether your disk is formatted NTFS, or want to know how to convert it, follow these instructions.
If you have important files stored outside of the above locations or you’d simply like to manage your own backup content, you can use the “Let me choose” option to customize which folders or drives will be included and whether to include a system image.
3. Choosing how often to back up your data
This screen of the wizard summarizes the backup settings, and let you select how often to run backup (or run it on demand). It is best to run backup on a schedule since you can truly “set and forget” – just make sure to schedule at a time that the computer will be turned on and the backup target is available.
When a system image is included, Windows will remind you to create a system repair disc. A system repair disc allows you to boot into your computer to access recovery tools or recover from a system image if Windows ever stops working. While these tools are available by default with your Windows installation, a system repair disc is needed if the hard disk fails. Note that a Windows installation disc can also be used in place of the system repair disc.
As you can see, in 3 easy steps, you’re done configuring backup and can now have the peace of mind knowing that your valuable files are well protected.
Ongoing data protection
Windows Backup runs automatically according to the configured schedule. If at the scheduled backup time the computer is asleep or the target is missing, that backup will be skipped but the next backup will still run according to the schedule.
The first backup created will be a full backup of all selected content, and subsequent backups will include only new or changed files (incremental backup). However, Windows Backup will occasionally create a new, full backup automatically if there has been many changes made to the files protected, such that you will have the option to delete older backups that might have become obsolete. We’ll come back to this topic when we discuss backup space management in a later post.
Once you’ve configured backup, you can find top level information on progress, status, and any relevant notifications through Action Center. The Backup and Restore control panel (accessible from Action Center or Control Panel), on the other hand, will provide more detailed information on your backup status and configurations and it is also where you can do other backup and restore tasks.
Making the most of Windows Backup
Windows Backup is simple to use, yet it provides a lot of flexibility on how you can protect your data and system. Here we’ll discuss a few tips on how to make the most of Windows Backup.
1. Choosing the right target
Windows Backup supports creating backup to internal/external hard disks, flash drives, optical discs, and network share (Professional and Ultimate Editions only). The biggest difference between these targets is the support for system image backup. A system image can be included in the scheduled backup configuration only if you are backing up to network location or hard disks, since space requirement makes it impractical to perform recurring backup to optical media. Also, while hard disks can store multiple versions of system image (newer and older backups), a network location can only store one system image per computer, meaning that as a new system image of your computer is created; the older version will be deleted.
|
Target
Feature |
CD\DVD\
USB flash drives |
Network location |
External\Internal
Hard disk |
|
Scheduled file backup |
P |
P |
P |
|
Include a system image in the scheduled backup |
- |
P
(only the most recent system image) |
P |
It is possible to create a system image to DVD on an ad hoc basis. We’ll discuss this in detail in an upcoming post specifically on system image backup.
Aside from the support for system image, other considerations may include factors such as price, amount of data to backup, reliability and security. For example, DVDs are light weight and inexpensive, but they may become corrupted over time and also become hard to manage as the number of DVD grows. While internal hard disks support the same functionality as external ones, you cannot store it in a location separate from your computer against disaster or theft. Therefore while we recommend that you save your backup on an external hard drive for the most flexibility, your target of choice may depend on your specific environment and need.
2. Organizing your important data using libraries
Library is a new feature in Windows 7 that provides a consolidated view of local folders located at various locations on the computer for easy access. This also provides a great way to organize your data for backup. Since Windows Backup backs up all local data in libraries by default, any new location added to a library will automatically be backed up without the need to reconfigure Windows Backup. For example, if you just created a new folder on your data drive for the family trip photos and include this folder under the Pictures library, it’ll be backed up automatically the next time Windows Backup runs. Alternatively, you can also create a “backup” library and add all your important data folders to it.
*Note that library folders that are residing on a network location will not be backed up.
3. Securing your backup
There are many ways a backup can be secured. It could be physical security (storing away the backup DVDs) or securing access rights (Windows Backup on hard disks and network share preserves user access controls of files). These are probably good enough measures for a home environment, but might not be enough if you’re on the go where your backup disk might be lost or stolen. In this case, you should secure your backup with BitLocker Drive Encryption (Ultimate Edition only).
You can use BitLocker Drive Encryption to encrypt the drive that you are saving your backup on, or to help protect the drives in your computer that you are backing up. To enable BitLocker, simply go to the BitLocker Drive Encryption control panel, and select “Turn On BitLocker” for the drives you wish to protect.
When a drive is locked by BitLocker, you need to unlock the drive before you can see information about the drive, back up the drive, or save a backup on the drive. Therefore if you’re using BitLocker with Windows Backup, the best option would be to set the drives that you are encrypting to unlock automatically when you log on to the computer. If you do not wish the drives to unlock automatically, you can also unlock a drive manually only when it’s needed for backup.
Your data is important, and Windows Backup is an easy way to help you protect them. It’s a good idea to set it up so you can spend your time exploring and enjoying Windows 7 and not worrying about losing your digital memories or documents. In the following weeks, we’ll discuss system image backup in detail, space management, and the data and system recovery experience of Windows 7. We hope these posts are helpful to you. Please feel free to provide feedback on materials you’d like to see covered or ask questions. We will roll them up into a FAQ at a later time if there’s interest. So post away!
-- Windows Backup team
In this blog post, let us explore how to install the DFS Replication Management Pack. System Center Operations Manager 2007 R2 is the latest release of the Operations Manager product. Check out this product comparison page for a quick overview of the new features in the SCOM 2007 R2 release. The DFS Replication management pack is also supported on the System Center Operations Manager 2007 SP1 release. The steps required to install and configure the management pack on SCOM 2007 SP1 are largely the same as those on R2 with the exception of a few usability improvements in the R2 release. For the purpose of this discussion, we will consider the SCOM 2007 R2 release.
For an overview of Operations Manager 2007, read this TechNet article.
A few Operations Manager concepts
The basic unit of functionality of all Operations Manager 2007 implementations is the management group. It consists of an installation of Microsoft SQL Server 2005 or Microsoft SQL Server 2008, which hosts the Operations Manager database, the root management server, the Operations console, and one or more agents that are deployed to monitored computers or devices.
Operations Manager Database: The Operations Manager database holds all the configuration data for the management group and stores all the monitoring data that has been collected and processed by the agents.
Root Management Server: The root management server (RMS) is a specialized type of management server in a management group, and it is the first management server installed in a management group. In brief, the RMS is the focal point for administering the management group configuration, administering and communicating with agents, and communicating with the Operations Manager database and other databases in the management group.
Agent: An Operations Manager 2007 agent is a service that is deployed to a computer that you want to monitor. Agents watch data sources (eventlog, performance counters etc.) on the monitored computer and collect information according to the configuration that is sent from the management server. The agent also calculates the health state of the monitored object and reports back to the management server.
Agentless monitoring: It is possible to monitor devices in an agent-less fashion. In this case, a management server performs the monitoring remotely.
Operations Console: The Operations console provides a single, unified user interface for interacting with Operations Manager 2007. The Operations console provides access to monitoring data, basic management pack authoring tools, Operations Manager 2007 reports, all the controls and tools necessary for administering Operations Manager 2007, and a customizable workspace.
Management Packs: Operations management packs (such as the DFS Replication management pack) contain best practice knowledge to discover, monitor, troubleshoot, report on, and resolve problems for a specific technology component. When imported into Operations Manager, they enable the agent to monitor the health of an application, generate alerts when something of significance goes wrong in the application, and take actions in the application and its supporting infrastructure to further diagnose the application or restore it to a healthy state.
For the latest Management Packs and Connectors from Microsoft and Microsoft Partners for Operations Manager 2007, visit the System Center Catalog at http://technet.microsoft.com/en-us/opsmgr/cc539535.aspx.
How Operations Manager works
The above (greatly simplified) diagram illustrates how the Operations Manager product works at a high level. System Center Operations Manager is installed on a Windows Server machine called the management server. The administrator can then select which computers are to be monitored. Operations Manager automatically deploys agents to these monitored computers if the administrator selects agent-based monitoring. For monitored computers that have been configured for agent-less monitoring, the management server performs monitoring remotely.
Administrators can import management packs such as the DFS Replication management pack using the System Center Operations Manager 2007 console. The console then deploys this management pack to the agents on the monitored computers. The knowledge contained in the management pack enables the agent to discover whether the DFS Replication service is installed, discover configuration settings and replication groups/folders etc. on the monitored computer. Further, it enables the agent to monitor the health of the DFS Replication service on these monitored computers, generate alerts if something significant goes wrong and help the administrator diagnose and restore replication to a healthy state.
Installing the management pack
Before installing the management pack, you need to set up the management server. Setting up the Operations Manager infrastructure is outside the scope of this blog post. The following pointers will help you plan and deploy your operations manager infrastructure:
The rest of this blog post and following posts explain how to install, setup and configure the DFS Replication Management Pack for Operations Manager 2007.
That’s it! You’re now ready to import the management pack using the Operations Manager console.
-------
Mahesh Unnikrishnan
The DFS Replication Management Pack for System Center Operations Manager 2007 is now available for download.
Download the DFS Replication Management Pack
| File Name: | DFS Replication Management Pack for Operations Manager 2007.msi |
| Version: | 6.0.6321.0 |
| Date Published: | 10/19/2009 |
| Language: | English |
About the Management Pack
The DFS Replication Management Pack for System Center Operations Manager 2007 monitors the health of the DFS Replication service on computers running Windows Server 2003 R2 or Windows Server 2008. This management pack retrieves events generated by the DFS Replication service that indicate the health of the service, replication groups, and replicated folders that are hosted on monitored computers. It also tracks important operational parameters such as the consumption of staging area and the number of replication conflicts generated. The management pack also includes a backlog monitoring view which provides an intuitive dashboard for monitoring replication backlogs.
Supported operating systems:
-
Windows Server 2003 R2
-
Windows Server 2008
Supported monitoring platforms:
Note: This management pack does not monitor the older File Replication Service (FRS).
Installation instructions
The following blog post contains instructions for installing the DFS Replication Management Pack.
-------
Mahesh Unnikrishnan
Don’t know how to get started on classification?
Afraid of getting it wrong?
Think classification is cost prohibitive?
Don’t understand what value you can get out of classifying and applying policy?
If you answered yes to one of the above – read on
How do I start? What if I make a mistake?
The Windows Server 2008 R2 File Classification Infrastructure makes it easy to start:
· Crawl, Walk, Run: There is no need to determine everything up-front. You can start with one classification property (e.g.: Secrecy=High,Medium,Low) or more and then add additional properties as you see fit
· It is very easy to set up – You can get familiar with classification in a Sandbox environment – using Hyper-V and a copy of data would be a great way to understand how classification and your actions will behave before working on the production environment
· You can “revert” classification mistakes: If you found an issue with the way classification was defined, just run a script to clear the mis-configured property and fix the configuration (see: http://msdn.microsoft.com/en-us/library/dd392357(VS.85).aspx)
Choosing classification properties
let’s talk about how to determine which classification properties you should use
Here’s a very simple guidance that we use:
Step 1: Determine the action that you want to apply to the data (e.g.: expire stale data, protect sensitive information …)
Step 2: Choose the right classification property for that action
An example – Reporting sensitive information on public file servers
Here’s a universal example: Making sure sensitive information is where it belongs. Let’s take an example where you would like to make sure you do not have any sensitive information on public file servers.
Your action would be to get a report of all the sensitive information that resides on public servers so that you can make sure to remove files that should not be there
To achieve this action you want to classify which files have sensitive information. For that, let’s choose a property name: “Secrecy” with potential values: “High”, “Medium”, “Low”
You can then set automatic classification rules to determine the classification of documents. For example – use content classification to mark all documents that contain the word “Confidential” as “High” secrecy and use folder classification to mark all the files that are placed in the engineering servers as “Medium” secrecy
Now, all you have to do is set a scheduled storage report of type: “report by property” for the “Secrecy” property so that it will be sent to your email once a week and provides the distribution of the “Secrecy” property on each of your public servers. If you see any “High” or “Medium” secrecy documents on these servers, you know that you need to take action
Most common actions
The three most common actions based on classification that we have seen and heard people use with FCI are:
|
Action |
Property (option 1) |
Property (option 2) |
|
Expire stale data based on time and classification |
None (just use file age and last modified) |
Retention: Long, Short, Indefinite |
|
Find and protect sensitive information on file servers |
Secrecy: High, Medium, Low |
|
|
Automated targeted upload of files from file servers to SharePoint |
None (just use file age and last modified) |
Project |
But these are all good material for additional blog entries J
Good luck with your classification
By Judd Hardy
It’s easy to see how the FCI platform helps you manage the files on your servers. In other articles, people have talked about how you can create properties to track and set up rules to have those properties automatically set based on file location or content. Then you can create file management tasks to apply company policy to a selected set of files with certain properties. For classification and file management jobs you can schedule them to run as often or as rarely as you want. In some scenarios, like file expiration, you can run the task at night so it doesn’t impact your server’s performance – it doesn’t generally matter if a file is expired precisely when it should be or eight hours later. In other scenarios, like protecting sensitive data, you probably don’t want to wait those eight hours – you need the file classified and policy applied to it as soon as the file gets copied to the serer. This is a good scenario for real time classification. The basics of real time classification are simple: watch for changed files, classify the changed files using the FCI APIs, then apply policy to those files that have certain property values. However, in reality there are a few extra details that you need to be aware of.
This article summarizes the “Near Real Time Classification using File Classification Infrastructure” code sample, located at http://code.msdn.microsoft.com/FCINRTC. I’ll start with the general outline of the sample and then go into details.
Near real time Classification, the overview:
- Track file activity on the file server
- Use a timeout to avoid transitional changes, such as Microsoft Word saving a temporary file (this makes it “near real time”).
- Process the changes for each file:
- Get the properties for the file.
- Skip files that were recently classified.
- Set properties on newly classified files and move them to the “recently classified” list.
- Apply the policy to the file.
- Cleanup the “recently classified” list.
Now, the details.
1 - Track file activity on the file server.
First, you need a mechanism to watch the file system and trigger events when files are created or modified - two actions that can impact a file’s classification. For this example we use the FileSystemWatcher class, http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx. Here is what’s involved in setting up your watcher. You set the path you want to watch and whether or not you want to include subdirectories. You set filters for what type of changes on what types of files will trigger your handlers. Finally, you add your event handlers to the watcher.
public void
Execute( )
{
// Create a new FileSystemWatcher object.
using (FileSystemWatcher watcher = new FileSystemWatcher())
{
watcher.Path = “d:\Share”;
watcher.IncludeSubdirectories = true;
watcher.Filter = “*.*”;
watcher.NotifyFilter = NotifyFilters.FileName |
NotifyFilters.DirectoryName |
NotifyFilters.Attributes |
NotifyFilters.Size |
NotifyFilters.LastWrite |
NotifyFilters.CreationTime |
NotifyFilters.Security;
// Add event handlers.
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.Created += new FileSystemEventHandler(OnChanged);
watcher.Renamed += new RenamedEventHandler(OnRenamed);
// Begin watching.
watcher.EnableRaisingEvents = true;
// Wait for the user to quit.
Console.WriteLine("Press \'Escape (Esc)\' to quit.");
while (true)
{
if (Console.KeyAvailable)
{
ConsoleKeyInfo key = Console.ReadKey();
if (key.Key == ConsoleKey.Escape)
{
break;
}
}
// Wait for some time and process files.
Thread.Sleep(1000);
ProcessFilesToBeClassified();
PruneClassifiedFilesCache();
}
}
2 - Use timeout to avoid transitional changes such as Microsoft Word saving a temporary file.
There are some things to note about the FileSystemWatcher class, mentioned here: http://weblogs.asp.net/ashben/archive/2003/10/14/31773.aspx, that will impact the design of our event handlers. The most crucial behavior for us is that multiple events can be raised for a single action. A similar scenario happens when a user saves the file frequently while working on it. We don’t want to classify the file each time the users saves it or each time we are notified by the watcher so we don’t immediately classify a file when the watcher signals us. Instead we put the file in a “to be classified” list. If the file is already in the list we update the timestamp of the file and move it to the end of the list.
private static void OnChanged(object source, FileSystemEventArgs e)
{
bool found = false;
foreach (FileItem fileitem in filesToBeClassified)
{
if (fullPath.Equals(fileitem.FullPath, StringComparison.CurrentCultureIgnoreCase))
{
found = true;
bool returnValue = filesToBeClassified.Remove(fileitem);
fileitem.TimeStamp = DateTime.UtcNow;
if (classificationAttempted)
{
fileitem.AttemptCount++;
}
filesToBeClassified.AddLast(fileitem;
break;
}
}
if (!found)
{
FileItem file = new FileItem();
file.FullPath = fullPath;
file.TimeStamp = DateTime.UtcNow;
file.AttemptCount = 0;
filesToBeClassified.AddLast(file);
}
}
3 - Process the changes for each file.
As shown earlier in Execute(), we periodically do two things to actually process the files we keep putting in our “to be classified” list. 1) we run through the “to be classified” list to see if any of the files are older than our update threshold, which is the minimum period of time to wait before actually classifying and applying policy to the file we put in the list. This allows us to consolidate multiple watcher events into one classification call. If the list has any files older than that threshold we classify and apply policy to them. 2) Any file that was classified in the previous step is placed in a “classified files” list and removed from the “to be classified” list. Once a file has been in this list longer that our cache threshold, we remove the file from the list. Keeping track of recently classified files allows us to skip classifying a file in the case where it is updated, but its classification doesn’t change. We don’t want to set properties if we don’t have to.
3a - Get the properties for the file.
Now, the first thing we do when a file crosses the update threshold is call IFsrmClassificationManager::EnumFileProperties to classify the file. The EnumFileProperties API can be run in two modes: run classification rules when determining the property value or just look at previously stored values, either from a previous classification run or properties embedded in the file. For our purposes we want to run the classification rules so that properties will show that weren’t there before. Also, if you have rules that use look at the content of the file you’ll want to have those run in the event that phrases you are looking for are added to the file. We also save off all the property definitions that are defined at the time the file is classified so we know what property values came from FCI and can be set later on.
private void
ClassifyFile(
FileItem file,
FsrmClassificationManager manager
)
{
IFsrmCollection propDefinitions = null;
// Enumerate all FCI classification properties of the
// file and save those properties with the file
// so they can be compared against future values.
file.Properties = manager.EnumFileProperties(file.FullPath, _FsrmGetFilePropertyOptions.FsrmGetFilePropertyOptions_None);
propDefinitions = manager.EnumPropertyDefinitions(_FsrmEnumOptions.FsrmEnumOptions_None);
file.PropertyDefinitions = new Dictionary<string, IFsrmPropertyDefinition>();
foreach (IFsrmPropertyDefinition propDef in propDefinitions)
{
file.PropertyDefinitions.Add(propDef.Name, propDef);
}
file.TimeStamp = DateTime.UtcNow;
}<
3b - Skip files that were recently classified.
Once the file is classified we check to see if the same file is in the “classified files” list. We not only check for its existence in the list, but also see if the property values are the same. If the file has already been classified and the classification hasn’t changed we skip this file. If the files was already classified, but the classification has changed we remove it from the “recently classified” list and treat the file as though it hasn’t been classified before. This allows us to see if the policy applies to the file now when it may not have previously.
private bool
IsFileInClassifiedCache(
FileItem file
)
{
bool isAlreadyClassified = false;
if (classifiedFilesCache.ContainsKey(file.FullPath))
{
FileItem classifiedFile = classifiedFilesCache[file.FullPath];
// Compare file properties.
if (file.Compare(classifiedFile))
{
// File properties have not changed. Update the timestamp of the file item.
classifiedFile.TimeStamp = DateTime.UtcNow;
isAlreadyClassified = true;
//
// Remove the file item from the files to be classified list.
// Note that, node may have moved in the linked list.
//
RemoveFileFromToBeClassifiedList(file.FullPath);
}
else
{
// File properties have changed.
bool returnValue = classifiedFilesCache.Remove(classifiedFile.FullPath);
classifiedFile = null;
}
}
return isAlreadyClassified;
}
3c - Set properties on newly classified files and move it to the “recently classified” list.
If the file hasn’t been classified or the classification has changed we set the properties for the file so it won’t need to be classified later during any automatic classification. EnumFileProperties doesn’t store any property values, even if you run classification rules and the file is newly classified, so it is important to set the properties in case any policy that is run changes the file, like RMS, or changes access to it so you can know why that policy was applied. One thing to note is we only set values for FCI properties. EnumFileProperties will return all the embedded properties for a file in addition to any FCI properties. That means that for Word files you get properties like Author, Title, etc. FCI can’t set properties that aren’t defined in it so we limit what we set to FCI properties. This is why we saved off the FCI property definitions with the file when it was classified.
private void
SetFCIProperties(
FileItem file,
FsrmClassificationManager manager
)
{
// Skip setting properties on ReadOnly files.
if ((File.GetAttributes(file.FullPath) & FileAttributes.ReadOnly) == FileAttributes.ReadOnly)
{
PrintMessage("File: \"" + file.FullPath +
"\" is a read only file. Properties are not set on read only files.", ConsoleColor.Yellow);
return;
}
foreach (IFsrmProperty property in file.Properties)
{
// Set this property on the file only if it is FCI property.
if (file.PropertyDefinitions.ContainsKey(property.Name))
{
PrintMessage("Setting property \"" + property.Name + "=" + property.Value +
"\" on file: \"" + file.FullPath + "\"", ConsoleColor.Green);
manager.SetFileProperty(file.FullPath, property.Name, property.Value);
}
else
{
PrintMessage("Found property \"" + property.Name + "=" + property.Value + "\" on file: \"" +
file.FullPath + "\"", ConsoleColor.Yellow);
}
}
}
3d – Move the file from the “to be classified” list to the “recently classified” list.
Once the file’s properties have been successfully set we add the file to the “recently classified” list and remove it from the “to be classified list.” Not much to say on this point, but important to do.
3d - Apply the policy to the file.
Now that we have classified the file and optimized to only act on those files where classification has changed we can get around to applying any policy we might want to take, provided the file matches our criteria. We loop through the properties we saved off when we classified the file (no sense in calling GetFileProperty and classifying the file again) and see if the value is what we are looking for. This sample only does a simple string comparison, but there are many other comparison operations you may want to consider. For example, if you are using an ordered list property you may want to act on all files whose value has a greater order value than a particular value. When you know the file meets your criteria you can send it to the exe or script that applies the policy. In this example we send the file to an exe that will encrypt it. Now we don't have to worry about our sensitive information sitting around unprotected until the file management job runs.
private void
EvaluatePolicyConditionAndExecutePolicyAction(
FileItem file
)
{
bool foundProperty = false;
// Get the specified property from the list of properties we already saved.
// We don’t want to call GetFileProperty for this.
foreach (IFsrmProperty property in file.Properties)
{
if (property.Name.Equals(propertyCondition[0], StringComparison.CurrentCultureIgnoreCase))
{
foundProperty = true;
PrintMessage("Found property \"" + property.Name + "=" + property.Value +
"\" on file: \"" + file.FullPath + "\"", ConsoleColor.Green);
// Check if the property value matches the value in the policy.
if (property.Value == propertyCondition[1])
{
// Replace [FILEPATH] string with the full path of the current file.
string commandArgs = commandArguments.Value;
if (commandArgs != null)
{
commandArgs = commandArgs.Replace("[FILEPATH]", "\"" + file.FullPath + "\"");
}
// Execute the command in separate process. Wait for completion.
PrintMessage("Executing command \"" + command.Value + "\" with arg \"" + commandArgs +
"\" on file: \"" + file.FullPath + "\"", ConsoleColor.Green);
Process process = Process.Start(“encryptfiles.exe”, commandArgs);
process.WaitForExit();
PrintMessage("Command return code:" + process.ExitCode, ConsoleColor.Green);
}
break;
}
}
if (!foundProperty)
{
PrintMessage("Property \"" + propertyCondition[0] + "\" not found on file: \"" +
file.FullPath + "\"", ConsoleColor.White);
}
}
4 - Cleanup the “recently classified” list.
The only thing left to do is prune old entries from the “recently classified” list. Here we remove any files that have been in the list longer than our “cache window” timeout.
private void
PruneClassifiedFilesCache( )
{
TimeSpan ts = TimeSpan.FromSeconds(cacheWindow.Value);
Queue<string> itemsToBeRemoved = new Queue<string>();
foreach (KeyValuePair<string, FileItem> file in classifiedFilesCache)
{
if (DateTime.Compare(file.Value.TimeStamp.Add(ts), DateTime.UtcNow) <= 0
{
itemsToBeRemoved.Enqueue(file.Key);
}
}
while (itemsToBeRemoved.Count > 0)
{
string key = itemsToBeRemoved.Dequeue();
classifiedFilesCache.Remove(key);
}
}
Summary
You might be tempted to just use near real time classification, let it call GetFileProperty and SetFileProperty, and always use it to apply policies to files - it's simple to implement and does the same thing as a file management job. However, you should remember to use the right tool for the job. Automatic classification and file management jobs are designed to efficiently scan whole namespaces where files already exist and are much more efficient than the GetFileProperty and SetFileProperty APIs. There’s also the chance that the FileSystemWatcher can miss files if there are too many file changes. For this reason it is a good idea to have a file management job configured for the same policy and namespace as your near real time-scripts to catch any missed files. Each approach to applying policy has its pros and cons, but if you have a share that you need to protect and can’t wait for a nightly task, near real time classification is the way to go.
Version 1.0 of the File Server Capacity Tool (FSCT) was announced yesterday during a presentation by Jian Yan and Bartosz Nyczkowski at SNIA’s Storage Developer Conference in Santa Clara, CA. The presentation covered a number of details about FSCT and included a demo running FSCT with the HomeFolders workload.
If you are not familiar with FSCT, the download page offers this overview: “File server capacity planning and performance troubleshooting are critical aspects of high-level network administration. Central file servers and distributed client workstations are now the norm in most corporate networks. This structure reduces storage capacity requirements, centralizes backup, increases the availability of files, and simplifies the document revision and review process. However, because data storage and access are centralized, performance limitations impact the entire network population. Accurately projecting the number of users that hardware can support under a specific workload, and understanding when and where bottlenecks occur, are critical to making efficient improvements to the server configuration. File server capacity planning tools can be valuable in choosing new hardware for purchase, identifying the capacity of existing hardware, locating existing bottlenecks, and planning for resource expansion in advance of resource exhaustion. The throughput capacity of a file server can be expressed either as the maximum number of operations per second or a maximum number of users supported by the configuration. These values are influenced by several factors, some of which include processor speed, available memory, disk speed, network throughput and latency, and the speed with which SMB requests are processed.”
The final version is available for download in both 32-bit (x86) and 64-bit (x64) versions. It is supported on Windows Server 2003, Windows Server 2003 R2, Windows Server 2008, Windows Server 2008 R2, Windows Vista and Windows 7 (with the latest service pack applied).
Downloads are available now:
x64: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=b20db7f1-15fd-40ae-9f3a-514968c65643
x86: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=0b212272-1884-4af1-972d-42ef1db9f977
A white paper is also included in the download packages, with detailed description of the tool, step-by-step instructions on how to use it and reference of the command line interface.
For questions about FSCT and how to use it, please use the forum at
http://social.technet.microsoft.com/Forums/en-US/fsct/threads
Also check some details on FSCT on these previous posts (from the release of the beta last year and the release candidate back in July):
http://blogs.technet.com/josebda/archive/2009/07/08/file-server-capacity-tool-fsct-release-candidate-available-for-download.aspx
http://blogs.technet.com/josebda/archive/2008/10/06/fsct-a-cifs-smb-smb2-file-server-tool-for-capacity-planning-and-performance-troubleshooting.aspx
This is a major milestone in the two-year journey to offer this tool publicly, which included efforts from a number of people from different teams at Microsoft, including the File Server Team and the Windows Performance Team.
Many customers have asked questions in our forums on how to understand and troubleshoot errors reported by Windows Server Backup. While we document the event IDs logged by Windows Server Backup (for example, see http://technet.microsoft.com/en-us/library/cc734488(WS.10).aspx), we do not document the error codes (HRESULT) displayed by Windows Server Backup. This series of articles clarifies these errors codes to help system administrators troubleshoot Windows Server Backup errors.
Windows Server Backup relies on multiple layers in the operating system to function. It can show errors when any of those layers report an error. If the origin of the error is inside Windows Server Backup, Windows Server Backup provides direct recommendations on resolutions. The challenge in troubleshooting is when the problem is in a layer not in Windows Server Backup, but which emerges and is shown by Windows Server Backup.
In this article, I’ll show you how to troubleshoot errors displayed by Windows Server Backup, when the origin of the error is in an underlying layer such as Volume Shadow Copy Service (VSS), Shared Protection Point (SPP), or other applications that plug into VSS framework. (For more information about troubleshooting VSS, see http://technet.microsoft.com/en-us/library/cc734243(WS.10).aspx )These issues map to following error codes: 0x8004*, 0x81001*, 0x807800A1, 0x8078006B, 0x8078008A and 0x8078001F-0x80780025. For a detailed list of these errors refer to Appendix A below in this article. This article applies to Windows Server 2008 and Windows Server 2008 R2.
Below are the steps that you should follow to troubleshoot errors where the cause is reported by an underlying layer to Windows Server Backup. Typically this involves using Event Viewer. To open Event Viewer, click Start, click Run, and then type:
eventvwr
General Troubleshooting Steps:
1. Check the event log for failure events logged by Windows Server Backup.
· How: Open Event-Viewer and review at the logs at: Windows Logs\Application where the source is Backup.
2. Identify which application or layer caused the issue.
· How: The event in previous step will usually indicate which layer or application caused the failure. For example, where VSS is the cause, you will see a message such as Check VSS event log for details.” .Check the event log of the problem application reported by the failure event.
3. Analyze events belonging to the application to understand the root cause. The events may point to issues in a different application.
· How: Search for events logged by the application found in step 2. For example, for errors caused by I VSS, this means, searching for all events where the value in the Source column is VSS.
4. Repeat step 2 and step 3 until you get to the root cause of the problem. Correct the problem.
Following this method to troubleshoot errors, I will walk you through a customer reported issue where we need to do two iterations to get to root cause. You can use this example to understand how to apply the steps above. Or, if you are lucky and get the same error as in this example, you can use it directly. J
Sample Issue:
When running Windows Server Backup, you receive the following error: A Volume Shadow Copy Service operation failed.
C:\TEMP>wbadmin start backup -include:d:,i: -backuptarget:f: -quiet
wbadmin 1.0 - Backup command-line tool
(C) Copyright 2004 Microsoft Corp.
Retrieving volume information...
This will back up volume NTFSTest(D:),Local Disk(I:) to f:.
The backup operation to F: is starting.
Creating a shadow copy of the volumes specified for backup...
The backup operation stopped before completing.
Summary of the backup operation:
------------------
The backup operation stopped before completing.
Detailed error: A Volume Shadow Copy Service operation failed.
Troubleshooting Steps:
· Step 1: Check the event log for failure events logged by Windows Server Backup.
On checking the Backup application event log, the following message is found:
The backup operation that started at '?2009?-?07?-?09T06:30:12.721000000Z' has failed because the Volume Shadow Copy Service operation to create a shadow copy of the volumes being backed up failed with following error code '2155348129'. Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
· Step 2: Identify which application or layer caused the issue.
From the backup event it is evident that the issue is caused by a VSS error.
· Step 3: Analyze events belonging to the application to understand the root cause.
On checking the VSS application event logs, the following warning event is found:
A VSS writer has rejected an event with error 0x800423f3, The writer experienced a transient error. If the backup process is retried,
the error may not reoccur.
. Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer.
Operation:
PrepareForSnapshot Event
Context:
Execution Context: Writer
Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}
Writer Name: Microsoft Hyper-V VSS Writer
Writer Instance ID: {25785983-e057-4b58-8d90-d0a7ef6c1a22}
Command Line: C:\Windows\system32\vmms.exe
Process ID: 10632
· Back to Step 2: Identify which application or layer caused the issue.
From the VSS event it is evident that the issue is caused by Hyper-V error.
· Step 3: Analyze events belonging to the application to understand the root cause.
On checking HyperV application event logs at Applications and Services Logs\Microsoft\Windows\Hyper-V-VMMS, the following warning event is found:
Failed to create the backup of virtual machine 'TestVM'. (Virtual machine ID XXXX)
· Back to Step 2: Identify which application or layer caused the issue.
From the Hyper-V event it is evident that the issue is caused by a virtual machine named TestVM.
· Step 3: Analyze events belonging to the application to understand the root cause.
On checking the application event logs inside the guest machine, TestVM, the following VSS event is found:
There is not enough disk space to create the volume shadow copy on the storage location. Make sure that, for all volumes to be backup up, the minimum required disk space for shadow copy creation is available. This applies to both the backup storage destination and volumes included in the backup.
Minimum requirement: For volumes less than 500 megabytes, the minimum is 50 megabytes of free space. For volumes more than 500 megabytes, the minimum is 320 megabytes of free space.
Recommended: At least 1 gigabyte of free disk space on each volume if volume size is more than 1 gigabyte.
· Step 4: Correct the problem.
The above error indicates that freeing up space on the physical volume hosting TestVM will solve the issue.
Appendix A
Windows Server Backup Error List
|
HRESULT |
HRESULT (in hex) |
Error Message |
|
2155348129 |
0x807800A1 |
A Volume Shadow Copy Service operation failed. Please check "VSS" and "SPP" application event logs for more information. |
|
2155347999 |
0x8078001F |
Windows Backup cannot find the shared protection point. |
|
2155348000 |
0x80780020 |
Windows Backup cannot find the shadow copy of the source volume. |
|
2155348001 |
0x80780021 |
Windows Backup timed-out before the shared protection point was created. |
|
2155348002 |
0x80780022 |
Windows Backup failed to create the shared protection point. |
|
2155348003 |
0x80780023 |
The on-disk writer metadata file is corrupted. |
|
2155348004 |
0x80780024 |
The specified Windows installation point does not contain a system ID file. |
|
2155348005 |
0x80780025 |
The operation to create the shared protection point was canceled. |
|
2155348075 |
0x8078006b |
Windows Backup failed to create the shared protection point on the source volumes. |
|
2155348106 |
0x8078008a |
Error: The read operation from the volume shadow copy on one of the source volumes failed. Make sure all the volumes included in the backup operation are available and accessible, and then retry the operation. |
VSS Error List
|
HRESULT |
HRESULT (in hex) |
Error Message |
|
2147754996 |
0x800423F4 |
The writer experienced a non-transient error. If the backup process is retried, the error is likely to reoccur. |
|
2147754806 |
0x80042336 |
The writer experienced a partial failure. Check the component level error state for more information. |
|
2147754995 |
0x800423F3 |
The writer experienced a transient error. If the backup process is retried, the error may not reoccur. |
|
2147754992 |
0x800423F0 |
The shadow-copy set only contains only a subset of the volumes needed to correctly backup the selected components of the writer. |
|
2147754754 |
0x80042302 |
A Volume Shadow Copy Service component encountered an unexpected error.
Check the Application event log for more information. |
|
2147754796 |
0x8004232C |
The specified volume is nested too deeply to participate in the VSS operation. |
|
2147754777 |
0x80042319 |
A writer did not respond to a GatherWriterStatus call. The writer may either have terminated or it may be stuck. Check the System and Application event logs for more information. |
|
2147754769 |
0x80042311 |
The given XML document is invalid. It is either incorrectly-formed XML or it does not match the schema. |
SPP Error List
|
HRESULT |
HRESULT (in hex) |
Error Message |
|
2164261121 |
0x81000101 |
The creation of a shadow copy has timed out. Try this operation again. |
|
2164261126 |
0x81000106 |
The backup process failed due to an internal error. For more information view the event log. |
|
2164261128 |
0x81000108 |
Windows cannot create a shadow copy on the specified disk. Please choose another disk. |
Note: The above error codes may change in future Windows releases. The messages above are for illustration purposes only and, depending on Window version, may not map to the messages shown in the actual product.
Post by Chirag Gupta
If you've been trying out the LUN Resync APIs with the Beta or RC bits for Server 2008 R2 you may have run into a few issues. Most of these have been addressed in the RTM build. Greg from the VSS team has a post detailing the fixes on his blog
-Dinesh
by Anupadmaja Raghavan
In the last post, http://blogs.technet.com/filecab/archive/2009/08/14/using-windows-powershell-scripts-for-file-classification.aspx, we saw how simple PowerShell scripts could be used to do custom file classification in Windows. This post will illustrate how PowerShell scripts could be used to do file classification based on the contents of the file. This post assumes that the reader has a good understanding of the basics of the PowerShell classifier module discussed in http://blogs.technet.com/filecab/archive/2009/08/14/using-windows-powershell-scripts-for-file-classification.aspx. The file classification infrastructure is referred to as FCI in this post (http://blogs.technet.com/filecab/archive/2009/05/11/classifying-files-based-on-location-and-content-using-the-file-classification-infrastructure-fci-in-windows-server-2008-r2.aspx).
The PowerShell classifier module provides the capability to read file contents to enable content classification. We saw in http://blogs.technet.com/filecab/archive/2009/08/14/using-windows-powershell-scripts-for-file-classification.aspx that for each file, during classification, the pipeline input from FCI to the PowerShell classifier module’s PowerShell script consists of an IFsrmPropertyBag object. The GetStream() method in the IFsrmPropertyBag object returns a standard PowerShell Stream object to read the contents of the file as raw byte streams.
In http://blogs.technet.com/filecab/archive/2009/08/14/using-windows-powershell-scripts-for-file-classification.aspx, we saw how to pass the mandatory parameter named ScriptFileName, from FCI to the PowerShell classifier module. The PowerShell classifier module is also capable of receiving additional parameters from FCI. This is done by specifying these additional parameters in the rule definition (similar to how we specified “ScriptFileName” in the rule parameters). The rule definition is an IFsrmClassificationRule object named $Rule which is available as a pipeline input to the PowerShell classifier module’s PowerShell script. $Rule.Parameters contains the fields specified in the Parameters section in the FCI rule.
Let us take an example to see how these two features could be used to do content classification. Say we want to classify files as either patents or copyrights of a company. Say the criteria we pick to classify files are based on the following known information:
- The copyright or patent documents are stored in *.txt files.
- The patent or copyright information is stored in the first line of the file.
- Patents contain the term “Patent of X” with the same casing.
- Copyrights contain the term “Copyright of X“with the same casing.
- One file cannot both be a copyright and a patent.
To do this using the PowerShell classifier module, we will create a String Property named “Document Type” and setup a rule to set this property based on output of the PowerShell script. The rule will be defined with the following parameters:
- Parameter Name: ScriptFileName, Value: <Name of PowerShell script file>
- Parameter Name: Copyright, Value: “Copyright of”
- Parameter Name: Patent, Value: “Patent of“
The PowerShell script will read the first line of any *.txt file and if it contains the defined terms above, it will classify the file with the “Document Type” property set to either “Copyright” or “Patent”.
| Process
{
################################
### Get the file name
################################
$PropertyBag = $_
$FileName = $PropertyBag.Name
################################
### If this isn't a .txt file don't process it
################################
if(!($FileName -like "*.txt"))
{
return
}
################################
### Collect the identifiers specified in the rule
################################
$Identifiers = @{}
foreach($RuleParam in $Rule.Parameters)
{
$Key,$Value = $RuleParam -split "=",2
If ($Key –ne ‘ScriptFileName’)
{
$Identifiers[$Key] = $Value
}
}
$FileStream = $PropertyBag.GetStream()
$FileStreamReader = new-object
System.IO.StreamReader($FileStream)
If ($FileStreamReader.EndOfStream)
{
return
}
$Line = $FileStreamReader.ReadLine()
$FileStreamReader.Close()
$FileStream.Close()
$Identifiers.GetEnumerator() | foreach-object
{
If ($Line.Contains($_.Value)) {
return $_.Key ### return the document type
}
}
} |
Thus the PowerShell host classifier module provides a simple way to do content classification of files using PowerShell scripts. More details on the topics discussed in this post and other capabilities of the PowerShell classifier module are available in the SDK.
Titus labs has posted some interesting blog posts that talk about FCI, SharePoint and how their product integrates with FCI
Microsoft Windows Server 2008 R2 File Classification Infrastructure (FCI)
Leveraging FCI in Windows Server 2008 R2
