I posted recently with some background information on the new AutoDiscover service in Exchange 2007.

Again, this is a service to help automatically configure Outlook 2007 profiles based soley on a user's email address.  This service also helps retreive the locations of new Exchange 2007 services such as free/busy, out of office, and web-based offline address book distribution.

For Internet Outlook 2007 users, the basics of this service are that they enter their email domain (name@emaildomain.tld) and Outlook automatically tries to connect to:

• https://emaildomain.tld/autodiscover/autodiscover.xml
• https://autodiscover.emaildomain.tld/autodiscover/autodiscover.xml

When retrieving settings from one of these URLs, HTTPS (SSL) is required -- which means there must be a valid security certificate installed for that web site that matches the site name. 

This, of course, poses an interesting problem for Exchange-based messaging hosters who do not want to have to buy a unique SSL certificate and set up a new web site for each new hosted domain.  (If they are hosting mail for mycrazywidgets.org, they don't want to have to buy a valid certificate for autodiscover.mycrazywidgets.org).  This also may be an issue for corporate environments that host multiple email domains but do not wish to purchase a certificate for each one.

Luckily, we have a solution for this -- that you can try out once Office 2007 Beta 2 Technical Refresh is released.  (Check here, I think, for when it is released).

There are a set of "one-time" configuration steps for hosters to get going, and then a set of steps for each new email domain that you host:

One-time configuration steps for multi-domain hosting & AutoDiscover:

1. Create a new virtual web site (on a new IP) that is Internet-facing.  Call it something like "autodiscoverredirect.[hosterdomain.tld]" where [hosterdomain.tld] is your 'main' domain name.  {The actual name of this virtual web site isn't really important}.  No certificate is required for this web site.
2. Create an /autodiscover/ virtual directory on that web site.
3. Create an empty file in this directory called "autodiscover.xml"
4. Through IIS manager, configure that file to be a redirect to https://autodiscover.[hosterdomain.tld]/autodiscover/autodiscover.xml.  (This can be set on the properties page of the file through IIS manager).

Per-domain configuration steps:

For each new hosted email domain

1. The DNS configuration of that email domain must be changed to add a CNAME record for "autodiscover.[emaildomain.tld]" pointed to "autodiscoverredirect.[hosterdomain.tld]".

Given that you already have to make DNS changes to host a new email domain (i.e., configure the MX record), this should just be one small additional step in that existing process.

Client experience

Now, what happens when a user types in emailaddress@[emaildomain.tld] into Outlook 2007?  This isn't the complete list, but Outlook will:

• Attempt to connect to https://emaildomain.tld/autodiscover/autodiscover.xml & fail.
• Attempt to connect to https://autodiscover.emaildomain.tld/autodiscover/autodiscover.xml & fail.
• Attempt to connect to http://autodiscover.emaildomain.tld/autodiscover/autodiscover.xml & succeed -- but receive an HTTP-level redirect to https://autodiscover.[hosterdomain.tld]/autodiscover/. 
• Warn the user about this redirect and ask them if they trust getting their settings from [hosterdomain.tld].  (The warning can be turned off by the user after the first time).  It says:  "Allow this website to configure user@domain.tld server settings?" followed by the URL of autodiscover at the hoster domain.  If the user does not recognize the hoster domain, then they should cancel.
• If the user accepts, Outlook will then connect to https://autodiscover.[hosterdomain.tld]/autodiscover/ and retreive profile settings.

One of my favorite new features in Exchange 2007 for end users is the updated Out of Office assistant.  In fact, this is a picture of me with the manager of our TAP program, KC Lemson, celebrating how much we love this feature in the Museum of Modern Art in NYC:

Why do I love this feature?  This post explains --

The rest is a guest post from Ashish Consul, a PM on my team:

The Out of Office (OOF) feature is commonly used by end-users to let other people know when they are not available to respond to e-mail.  Exchange 2007 Out of Office capabilities such as scheduled OOF, different external and internal OOF messages and the ability to control what kind of OOF to send on a per-domain basis improves the experience for both end-users and for administrators.

For End-Users

In Exchange 2007, some of the new Out of Office Assistant features are:

· The ability to schedule OOF messages in advance.  If users are planning time away from the office, they can set their OOF to begin at a future date and time and do not have to worry about forgetting to turn on OOF right before leaving. 

· Separate internal and external OOF messages.  This way, private information can be shared with co-workers without sending that information out to external senders. 

· External OOF messages can be sent only to a user’s external contacts.  Some users want to send their Out of Office messages only to a limited set of external contacts, but not anyone who might email them – for privacy reasons. 

• OOF messages can now be set as HTML instead of plain-text.

OWA and Outlook 2007 users who are hosted on Exchange 2007 mailboxes will see these new capabilities.  Outlook 2007 users who are on Exchange 2003 (or earlier) servers will have the same features as they have with Outlook 2003. 

This screen shots show the OOF configuration options available to end-users in Outlook Web Access:

For Administrators and Exchange organizations

In Exchange 2007, Administrators can choose to control external OOF capabilities at a per-domain and per-user level. 

Some organizations do not enable external OOF messages today, because they do not believe they are “safe”.  Some users put personal information in their OOF message that should not be leaked externally, and some companies have expressed concern about OOF messages as a mechanism for spammers to validate users’ email addresses.  In Exchange 2007, we believe we’ve addressed this organizational concerns.

For other customers, they would like to enable some of their users for external OOF messages (such as the ones working directly with customers), but not all of them.

Per User Settings

Exchange 2007 lets administrators control per-user external OOF messages using the Monad command “Set-Mailbox” with the “ExternalOOFOptions” parameter:

MSH>Set-Mailbox –id <mailbox identity>  -ExternalOOFOptions [InternalOnly,External]

By default, per-user external OOF option is set to allow external OOF.  By setting this to “InternalOnly” for a given mailbox, instead of “External”, that mailbox will not be able to send OOF messages outside the company, using any client, regardless of what the user selects.

Per-Domain Settings

Administrators can also control which (if any) OOF messages go to which external domains.  These per-domain settings are set via the Monad command  “Set-remoteDomain” using the parameter “AllowedOOFType”:

MSH>Set-remoteDomain –Identity <domain identity> -AllowedOOFType [None,InternalLegacy,Internal,External(Default)]

The AllowedOOFType values are as follows:

· None:  This blocks all OOF messages to the target domain.

· InternalLegacy: This allows either Exchange 2007 internal OOF or OOF set by Outlook 2003 or earlier clients to be sent to the target domain but blocks Exchange 2007 external messages from being sent to the target domain.  You would use this setting with a close partner company who you wanted to receive your internal OOF messages.  You would also use this setting in a cross-forest environment where one company had multiple Exchange deployments.

· ExternalLegacy: This setting allows either Exchange 2007 external OOF or OOF set by Outlook 2003 or earlier clients to be sent to the target domain.  If you already allow OOF messages to flow outside your company, you probably want to use this setting, so that existing Outlook 2003 users do not see any change in behavior.

· External(the default):  This setting allows only Exchange 2007 external OOF messages to be sent to the target domain but blocks internal OOF messages or OOF messages set using Exchange 2003.  If you currently block OOF messages to other domains, you probably want to use this setting in order to keep the same behavior for Outlook 2003 users but allow Exchange 2007 external OOF messages out for those who specifically choose them.

You can choose the wildcard ‘*’ for the target domain if you wish your per-domain OOF settings to apply to all external domains that don’t otherwise have a specific setting for them.

The following screen shot shows the per-domain OOF configuration settings in Exchange Management Console:

In addition, in Exchange 2007 OOF responses are more secure because of the following two checks:

· OOF messages will not be sent out in response to server-detected Junk E-mail.  If Exchange 2007 detects a message as Junk or if the message sender is in the user’s block list, no OOF response is sent. 

· OOF messages are not sent as responses to Internet mailing lists.  Exchange 2007 does not send OOF responses for incoming messages with the Precedence:bulk header set.  However, if this header is missing, it is possible that OOF responses may be sent out because there is no reliable way of determining that the message was sent to a mailing list.  OOF messages are also not sent as responses for incoming messages with the X-Auto-Response-Suppress:OOF header.

One of the features that my team is especially proud of for Exchange & Outlook 2007 is the new "Scheduling Assistant" that is in Outlook Web Access and Outlook 2007. If you want to see a live demo of this functionality (and all of Exchange 2007 calendaring) you should take a look at this TechNet Webcast that Paul Tischhauser, PM for the feature, did. Anyway, I thought it might be interesting to talk a little bit about how this feature evolved over time – starting from an idea to its development & completion.

Purpose of the "Scheduling Assistant"

We knew that calendaring was going to be one of our major points of investment in Exchange 2007. In the fall of 2003 and spring of 2004, we visited a number of companies to observe information workers, learn their pain points, and find out how they interacted with the product. One of the things that kept coming back to us was that – calendaring was too difficult, it takes too long to set up a meeting, and nothing has improved in Outlook in this area since it was first shipped. Since our team was tasked with making Exchange 2007 the best server for Outlook users, we wanted to figure out what we could do – across both server and client – to improve this experience.

We knew, from experience and from talking to users, that the "AutoPick Next" functionality in Outlook 2003 really wasn't that useful for scheduling meetings. For people that had busy schedules, and were trying to schedule a meeting within the next week or two, using this button would certainly find a time for the meeting – but it would be three months away. We wanted to figure out the right way of balancing all of the various inputs and requirements that go into meeting scheduling, including:

• The "importance" of various attendees – the Executive VP is really required, but if Jon the Intern can't make it, that's ok. (On the other hand, for Jon the Intern's going-away party, the VP isn't really required – but Jon is).
• Requirements around conference room booking – location, capacity, available equipment (PolyCom phone, Ethernet port), permission to book, availability of the room, etc.
• The time frame of the meeting – often a meeting next week that 75% of the people can attend is more valuable than a meeting in 3 months that everyone can make

So we started brainstorming on various ideas around how to simplify the scheduling experience, and put together some prototypes. I was originally a big fan of doing something similar to the flexible search options found at Orbitz.  (I like how they let you be specific, yet flexible, about your requirements – and how they present search results in a very powerful way. Three years later, I still think it is the best airline search out there). In the end, we went a different direction – based on user feedback – and came up with a design that enables users to slowly transition into a new way of thinking about scheduling without taking away what they like about the current design.

Round 1 – Conference Room search & Booking

Paul and I worked closely with one of the excellent members of our design team, Michael, to come up with some prototype ideas to test in front of users. For one of the first iterations of our design, we looked just at the issue of booking a conference room. First you would enter in the criteria for your conference room. This could include things like equipment you needed, the building the conference room was in, and the timeframe that you needed this in. Then, you would get back search results that would show you available time slots, and a drop-down list of rooms to choose from that were available at that time. We knew that some conference rooms will always require approval before booking them, so we wanted to also include that information in the results list. This is what that looked like (click on it for full detail):

This produced what I still think is some nice looking UI, but there were really two problems with this design:

• It wasn't integrated into the Outlook booking process – this was designed almost as a stand-alone app and didn't fit well into Outlook's meeting request form & model.
• It put scheduling resources (conference rooms) before scheduling people. As we found out, that isn't really how people work. Users tend to find a time that everyone (or most everyone) can meet first, then they find a conference room. But the most important thing is the people, not the room.

Round 2 – Integrated into Outlook; People AND Resources are important

This next design shows integration into Outlook – as a new tab on the meeting form called "Smart Scheduling." (The "tab" design was before Outlook had implemented the "Ribbon"). In this design, you still specified your search criteria explicitly for your meeting. You could provide "flexible" or "specific" search criteria (my influence from liking Orbitz) for both the time of your meeting as well as the resources required for your meeting. I still kind of like this design, although it would have been more complex than our final implementation. Here is the first time you see the meeting options listed on the right-hand side in a format that is fairly close to what we ended up with. (Click on image for full detail).

We tested this type of design in front of users, but the feedback we got is that they missed seeing the individual attendee's free/busy, like they could in the normal scheduling tab. Although users appreciated our meeting options as a quicker way to find a good time to meet, they weren't comfortable with it unless they got to "see the data themselves" to confirm that our "smart" suggestions were really doing the right thing.

This is when we determined we needed to figure out a way to deliver the best of both worlds – in a way that enabled users to slowly transition from their current free/busy-based way of picking meeting times to using our new model.

I'll talk about how this continued to evolve in a subsequent post…

Part 2: When we last left our superheroes (see Part 1) -- the Exchange Mailbox IW team -- they were still iterating through potential scheduling form designs to come up with the final plan.  (Click on all the images below to see the full size version).

Round 3 – The return of free/busy!

This design starts to get much closer to the final plan. After putting a number of the other designs in front of real users (yes, we actually do this sometimes), they told us loud and clear: we like the meeting suggestions as 'helpful hints', but we still want to see the free/busy. Basically, in order to get users to trust the new suggestions, we also had to give them a view of the original data (free/busy) so they could double-check that we were making smart suggestions. This lead us to the first view that integrated free/busy and meeting suggestions on the same tab.

There are a couple of other things to point out about this design:

• In this design, the "People" section and "Resources" section were split, and overall people preferred this design. It makes a lot of sense because users think about scheduling people differently than they think about scheduling rooms. We were able to get this "split grid" design into OWA (because it was being re-written anyway) but unfortunately weren't able to get it into Outlook due to time constraints.
• The "Location" drop down for Rooms, in this design, is something we should revisit in the future. The idea here was that you could quickly select a location and all of the rooms in that location would be added to your scheduling form (so that one free one could be chosen). We didn't end up getting a completely robust concept of 'location' built into the system, so this got dropped. However, OWA retained part of the concept in the final version with the ability to easily add all 'recently used' rooms.
• In "Round 2", the timeframe for the meeting suggestions was set through a dropdown; in this design, we introduced a calendar control that would pop up in front of the meeting suggestions when you clicked on the tab above it. The idea of the calendar control was that you could select one-by-one individual days that were 'ok' for you to meet in – so that the system would only provide suggestions during those particular days. The orange days you selected were the ones that you wanted suggestions generated for. The other interesting control in this design was a "Time of Day" filter (not shown active). If you clicked on "Time of day", a different control would be shown that would enable you to filter which particular hours of the day you'd like to have the meeting. The "Time of Day" plus "Timeframe" controls together would serve to filter the meeting suggestions that were returned.

Round 4 – Remove the 'time of day' and 'timeframe' controls

At some point, it dawned on us that instead of using the calendar control as a way for the user to filter results, that there were better ways to use it:

• The calendar control could be used to navigate the free/busy grid. One problem we heard and observed when we had talked to users is that it was quite easy for users to get "lost" in the free/busy grid and lose track of which day they were actually viewing. They didn't have an easy way to move around in the grid. We thought the calendar control could be used by the user as a free/busy navigation aid, and it turned out that worked quite well. This made it into the final version.
• In "Round 3", we were asking the user for a lot of input – like the timeframe of the meeting, and their preferred meeting time. Although this allowed for discrete control over the meeting suggestions that were returned, it wasn't clear that the benefits outweighed the cost of asking the user for more information. For one, instead of asking the user for the 'time of day' that they preferred the meeting, we could simply use the working hours already configured in Outlook. And instead of making them choose which days they wanted to potentially meet, we could just show them all the days and then color the ones that were better days for meeting vs. others.

This form also shows equipment called out as a separate row on the scheduling tab. This didn't make it into the final design – and improving equipment scheduling is something we should think about tackling in future releases. [The more I work in this area the more I am convinced there is so much for us to do still. Email and scheduling are far from being 'done'].

Round 5 – Prototype

This is one part of the effort that I'm personally proud of – and also helps to show how it is useful for Program Managers to be able to code up something every once in a while. Before we made the final no/no-go decision on building the feature in Outlook and Outlook Web Access, it was important to see the interaction "in action" to get a feel for how – and if – it could work. I built a prototype in ASP.NET of the design which – except for formatting – you can see looks very similar to the final design in Outlook Web Access. Since we were NOT part of either the Outlook or Outlook Web Access teams, building this prototype helped us have something we could show to them to prove its feasibility.

Round 6 – Final design

This is a screen shot of what it finally ended up looking like in Outlook Web Access. (Outlook looks very similar, except attendees and rooms are not in separate sections of the free/busy grid). In this example, I can see that August 8^th-22^nd are not good days to try to meet with these two other people but other days in August should work fine. I've also added a bunch of potential conference rooms to the form. By doing this, I can click on one of the meeting suggestions on the right – and choose a particular conference room. (No longer do you have to add a bunch of conference rooms, find the one that is free, and then delete the rest off – if you use the suggested times list, this gets taken care of for you).

In fact, the system works best when you intentionally add multiple conference rooms to the form. This way, the suggestions can tell you which room is free for that particular time and select it for you if you click on that suggestion. Intentionally adding multiple conference rooms to the form is a change that users should make to get the most out of this form.

Conclusion

I hope this was an interesting look into how one particular feature got built in Exchange 2007. If you have ideas on how we can improve the scheduling process even more in future releases, leave me some comments. We should also be posting a video walk-through some time with the PM for the feature (we've already filmed it).

Need help troubleshooting Outlook 2007 / Exchange 2007 "AutoDiscover" and free/busy features?

We've started to populate the Exchange Wiki (http://www.exchangeninjas.com/) with some Exchange 2007 related information. 

A PM on my team posted instructions on understanding and debugging Outlook's use of web services in E2k7.  http://www.exchangeninjas.com/AvailabilityServiceFAQ.  Its a good start and has some information that is not in current versions of the "official" documentation.

Some background --

Many moons ago when we began the process for planning Exchange 2007, we had a goal of reducing TCO for Exchange (well, this is really a perpetual goal).  In our investigations, we discovered that the top corporate helpdesk expense -- in supporting Outlook & Exchange -- was helping users to configure their Outlook profiles.  Even I, as a member of the Exchange team, have had to call the helpdesk before in order to configure Outlook Anywhere (RPC over HTTP) -- because I just didn't know the server name.

Out of this was born the idea of creating an "AutoDiscover" service -- a way for Outlook users to get automatically connected without having to manually type in a bunch of server settings.  The approach was chosen of having clients connect to autodiscover.[my email domain].[tld] in order to retreive an XML file containing their profile settings.  All the end user would have to know was their email address and password -- pure simplicity.  In addition to the mailbox server, this mechanism was also used to help Outlook "discover" other new Exchange 2007 services that it could connect to -- including our new free/busy, new OOF assistant, Unified Messaging settings and the Offline Address Book.

While this approach of using  autodiscover.[my email domain].[tld] was reasonable for Outlook clients connecting from the Internet, we soon discovered it wouldn't work from within many corporate environments -- a good percentage of which do not enable Outlook access from the Internet anyway using RPC/HTTP.  Furthermore, there was a poor "out of the box" experience -- if you installed Exchange, and then installed Outlook 2007 -- it didn't "just work" -- you had to fiddle with DNS and certificates instead.  Sometimes, the people configuring Exchange didn't even have access to the corporate DNS to make the necessary changes to create autodiscover.[my email domain].[tld].

We needed a solution that balanced the necessary security when connecting over the Internet with the simple out-of-the-box experience that most locally-connected, domain-joined users should have. 

Exchange had a plan of automatically generating and installing self-signed SSL certificates when it was installed.  Although these certificates cannot be fully 'trusted', the traffic to those servers is then automatically encrypted -- which is certainly better than nothing at all. 

The conflict was -- when connecting over the Internet, we knew that autodiscover.[my email domain].[tld] should require a valid, trusted SSL certificate -- or else it could be spoofed, and you could accidentally try to authenticate against a bad, bad server and get bogus server settings.  Because we had these self-signed certificates, Outlook would not connect at all to Exchange without installing the certificate on the client machine or setting a regkey that would cause Outlook not to use SSL.  While this could be viewed as a security 'feature' when connecting in over the Internet, it was a real pain when you were inside the corporate network and just trying to get your brand new Outlook 2007 client to work properly with Exchange 2007. 

The fix....

Luckily a smart PM lead (not me) dug around and came up with a solution using a capability of the Active Directory (AD) that we hadn't realized existed -- something called a Service Connection Point. (SCP).  This was specifically created to help client applications locate particular services within an AD forest. 

The decision was then made to use this SCP thing to help Outlook find an AutoDiscover server without having to use autodiscover.[my email domain].[tld].  If Outlook was on a client machine that was part of an AD forest, and it could contact a Domain Controller -- it would do an LDAP query to find an SCP for Exchange AutoDiscover and then use that URL to connect.  In this case, because we definitively know that Outlook is running on the corporate network, Outlook would ignore SSL certificate errors when trying to connect to Exchange web services. 

If Outlook was unable to contact a domain controller, it would fall back to trying to connect directly to autodiscover.[my email domain].[tld] -- the method that does require DNS configuration.  In this case, valid & trusted SSL certificates are required to prevent any sort of server-spoofing on the Internet.

Still having problems?

The scheme described above made it into Exchange 2007 Beta 2 but was too late for Outlook 2007 Beta 2 -- so you won't see a great out-of-the-box experience with Outlook 2007 Beta 2 until the "Technical Refresh" of Beta 2 is published.  But when it is, you should try the two out together and let me know if this seems to fix the issues for you.

And hopefully, some of the information here (http://www.exchangeninjas.com/AvailabilityServiceFAQ) can help you get going in the right direction.