I noticed a thread on one of the Exchange yahoogroups a few weeks ago about how the "Enable FBA" checkbox was grayed out on a certain server. It turns out that this is the expected behavior on clustered servers (and as a workaround, can use a front-end to connect to the BE cluster, and enable FBA on that front-end). Based on that conversation, I wrote a KB article about this, and it's now public:

http://support.microsoft.com/?id=834637

Yes, we do listen.