Removing DFSR Filters

Removing DFSR Filters

  • Comments 11
  • Likes

Hi folks, Ned here again. DFSR administrators usually know about its built-in filtering mechanism. You can configure each filter based on file name and extension; by default, files named like “*.bak, *.tmp, ~*” are filtered, as they are unlikely to be permanent or useful between servers. You can also filter out folders; this is less common, as you cannot provide a full path - only a name. Sometimes though, it is useful for a specific working folder used by applications.

When you enable a filter, each server scans its database for that replicated folder path and removes any records of files and folders that match the filter. Because the database doesn’t care about filtered objects, DFSR ignores any future changes to the files. In practical filtering terms, this means:

1. Any new files/folders added do not replicate to any other server.

2. Files/folders that already exist through previous replication stay on all servers.

3. Files/folders previously replicated and then later modified after enabling the filter are quasi-deleted from all other servers, in the sense that they are added to the database tombstone list. After all, they are filtered and therefore “no longer exist” when updated, according to the downstream servers. But the files do not actually get deleted from the replicated folder; they simply stop replicating and if anyone changes them on various server nodes, they diverge.

But what about when you remove filters?

image

This is rare enough that we've never bothered to document it. There are key issues to understand:

1. You must install the latest DFSR hotfixes for your operating system, on all DFSR servers.

List of currently available hotfixes for Distributed File System (DFS) technologies in Windows Server 2008 and in Windows Server 2008 R2 - http://support.microsoft.com/kb/968429

List of currently available hotfixes for Distributed File System (DFS) technologies in Windows Server 2003 and in Windows Server 2003 R2 - http://support.microsoft.com/kb/958802

Do not continue with any filter changes until installing those hotfixes and restarting the DFSR servers*. There is a very nasty bug that leads to folders refusing to replicate their previously filtered-contents or files that disappear from partner servers.

* Alternatively, you can stop the DFSR service and install the hotfixes. Generally, the removes the need to restart and no prompt displays.

Note:

I'm often asked if DFSR hotfixes are recommended preemptively and the answer is YES. Data loss hotfixes do not fix your lost data, only make further data loss stop! You should probably keep on top of NTFS hotfixes too.

2. Any changes made between servers after enabling the filter are going to result in some deleted or overwritten files, and that is by design. By setting a DFSR filter, you told DFSR that those folders and their contents no longer existed in DFSR terms. When the filter comes off and after making the first change in some server’s copy of that folder, the conflict resolution algorithm is going to kick in and the two folders synchronize, regardless of your wishes as to which folder will win.

Here I filtered subfolder2 and created – on each server – different bmp files named madeon1 (on server 01) and madeon2 (on server 02):

image

image

Then I removed the filter, forced AD replication, and forced DFSR to poll AD. Those files are different, so nothing “bad” happened and they sync:

image

However, if I repeat that un-filtering test with two files with the same name, but different contents (the file was originally replicated to both server, filtered out, and then later modified on server 02):

image

image

Then the newer file will win, overwriting what’s on 01 (even if it was “good” data for some user). This is why when you filter folders from replication, you should make sure it’s not user data that is still modified on multiple servers. Someday it may have to re-converge.

image

3. It is critical that you back up filtered folders on all servers replicating its parent, as you are very likely to need to restore some files in order to undo some of the conflict resolution damage. Some users are going to be very unhappy otherwise – and if they are the Vice President of TV Programming and Microwave Oven Technologies, they will come down on you like a load of bricks.

A minority of customers use DFSR folder filters – they are not very flexible, due to their lack of path support. They are safe only if users cannot alter them or their contents – then at least they will not have a negative experience.

Until next time,

- Ned “Director of Directories” Pyle

  • I love DFSR filters ( file nobody uses folders!), the algorithm while adding new filters is tremendous improvement on FRS, right?

  • Hey look, the canary got an upgrade to Canary 2.0!  Much more minimalist, and actually looks a little like you were influenced by John Lennon...

    http://www.lennonart.com/

    :)

  • Now I have a fallback career in case this whole computer thing doesn't pan out.

  • I think it is a pitty that in specific circumstances, DFSR looses data. We are looking into DFRS as a solution to bring data from remote sites to a central location that is backed up in a decent way.

    The more I read, the more I am convinced DFSR will not enable us to make less backups, instead we will need more :(

  • Dude, if you say “You must install the latest DFSR hotfixes for your operating system” then KB968429 is not the very best source since it's constantly outdated. Way better link is www.bing.com/search

    Well, Here's a more readable result for those who are too lazy for Bing: social.technet.microsoft.com/.../5801.aspx

  • @SenneVL - well, they were bugs. It's not like it was intentional or expected. Based on that logic, you should not be using NTFS either, as it has data loss bugs. All file systems do though.

    @ Artem -  That's a cool link, very good job. The KB being out of date is a constant battle, (ongoing as we speak). In this case though, it includes the necessary updates to prevent the issue.

  • Updated my last reply to be more useful. :) I might also suggest adding the other DFSR-related binaries mentioned to your wiki post; that's the nice thing about the KB: it covers all related files, such as minimum NTFS.SYS.

  • Technically, what's included in updates are not arbitary files, but CBS packages. So, as stated on the Wiki, the “Microsoft-Windows-DFSR-Core-ServerOnly” includes dfsrprovs.mof, dfsrress.dll and dfsrs.exe. What it means is that even when we need to update any single of them, the same (read: latest to date) versions of all three are installed. (This description is somewhat simplifed but I suppose it's meaningful and relevant).

    Bottom line: if you suggest any other DFS-R files/packages, we can try to maintain Wiki tables (with Bing links) for all of them. And yes, your NTFS.sys suggestion is heared.

    And if you need more votes for you constant battle, you can count me. It's one of my favorite topics over years (even since I was an MVP).

  • Awesome, thanks. Anything listed on the DFSR hotfix KB is relevant to DFSR, so if you wanted to expand this wiki post to become "all DFSR-related QFEs" that's all it would take.

  • I've just run into this issue - I removed the default filters but they're still active which is causing problems with office files (because the ~$ lockfiles aren't replicating, it's allowing concurrent open files).

    Is there a recommended order to apply the hotfixes?  e.g. should I recreate the filters or will applying the hotfixes alone be enough to fix the issue.

  • Installing the hotfix does not repair any existing issues, it just prevents further issues (at this point, your database has goofed-up entries). To fix the issue on the existing files, you should open a support case; it may require rebuilding the database. You can *try* making a trivial change to the ~ files manually, such as adding a harmless auditing ACE.

    However - this is probably immaterial. In my repro with Word 2010 and letting it successfully replicate the ~ lock file, even after the ~ file was deleted by the original user closing Word and the ~ file no longer existed anywhere, Word always opened the replicated copies in *read-only mode*. So that remote copy is seemingly permanently read-only and totally FUBAR, even after I made further edits that replicated to the other server without issues. I was forced to save as a different name, then delete the original and rename my save to the old name. This is worse than the default behavior by far. So while I'd still add the hotfix, I would not remove the tilde filter.

    This is something within Office storing the state of a file; it's not even the user profile - a brand new user saw the same problem on replicated data he was accessing for the very first time. Very odd and pretty yuck. I had no such issues when DFSR was using its default ~ filter. There is no real chance that the office team is testing your proposed config (it's unlikely they ever heard of DFSR, honestly).