Team blog of MCS @ Middle East and Africa

FAST Search for SharePoint 2010 - FQL and its Impact on Relevancy Ranking (Using Full Text Index Mapping)

Ahmad Shahin — Fri, 24 May 2013 16:48:00 GMT

Introduction

In this post I will describe a common mistake that occurs when you write your own FQL that impacts the search results that depends on Relevancy Ranking (Using Full Text Index Mapping).

Scenario

Lets consider that we have a list of products that we would like to search through that have a custom managed property weights defined to control Relevancy; The Full Text Index Mapping is configured as following:

ProductName: Level 7 (High Importance)
ProductDesc: Level 1 (Low Importance)

and lets say that other Dynamic Ranking options are disabled (such as Freshness, Boosts, Authority Weight ... etc.) in this scenario to make things easier. So what happens if I searched for the product "XBox" using the following FQL:

(

ProductName:string(

"XBox",

mode="simpleany"

)

) or (

ProductDesc:string(

"XBox",

mode="simpleany"

)

What I expect to see is that the search results will bring the result items that has the word "Xbox" in the ProductName first due to my relevancy configuration, but that doesn't happen, why?

The Problem:

To understand the problem lets check the official definition of FQL from Microsoft MSDN Site:

The FAST Query Language (FQL) is a powerful query language that enables developers to perform exact searches and to narrow the scope of your search to values that belong to a specific managed property or a full-text index.

So what happened here is that the Exact Managed Property Search kicked in and disabled the Dynamic Ranking (using Full Text Index Mapping).

Solution

To solve this issue you need to remove the managed properly names from your query and rephrase it as following:

(

string(

"XBox",

mode="simpleany"

)

This will enable the Full Text Index Mapping on your reach results and brings in the results that are ordered in the ProductName first.

Note:

For the sake of this scenario I disregarded the number of occurrence of the word "Xbox" in the search results to make it easier to understand, please note that this issue will also affect the order of the search results.

Additional References

More useful information can be found in the below links:

Index Schema - http://msdn.microsoft.com/en-us/library/ff464344(v=office.14).aspx#schema_concepts_full_text
Ranking & Sorting - http://msdn.microsoft.com/en-us/library/ff394654(v=office.14).aspx
FAST Search for SharePoint Relevancy Post by Eric - http://fs4sp.blogspot.ae/2012/01/fast-search-for-sharepoint-relevancy.html
FAST Query Language (FQL) Syntax Reference - http://msdn.microsoft.com/en-us/library/ff394606(v=office.14).aspx

Diagnostics of Cloud Services – Custom Trace Listener

Mecit Atmaca — Thu, 23 May 2013 20:55:12 GMT

In this part of the series, I am going to talk about instrumenting diagnostic features for cloud services (web and worker roles) with a custom trace listener implementation in respect to new Windows Azure SDK for .NET (April 2013, version 2.0).

Before starting the implementation, I would suggest you to get an overview of the new SDK. A perfect resource for this would be Scott Guthrie’s blog here. As you can see, one of the areas improved noticeably since v1.8 is diagnostics and key items of which are:

Built-in diagnostic features for Azure web site enhancements

Being able to diagnose cloud services directly from Visual Studio

Being able to change diagnostic configuration settings while application running.

In .NET world, to diagnose and monitor an application, you need to implement a listener that listens the source which is the application itself for events (debug, trace, etc.) and a writer that persist the messages that captured through the listener to intended path (file, database table, storage table, etc.).

Listener

All (custom or built-in) listeners are derived from System.Diagnostics.TraceListener. I am going to implement a table storage listener, as name implies, it will outputs the captures to an Azure storage table. Here is the whole class implementation:

        1: using System;

       2: using System.Collections.Generic;

       3: using System.Data.Services.Client;

       4: using System.Text;

       5: using System.Diagnostics;

       6: using Microsoft.WindowsAzure.Storage;

       7: using Microsoft.WindowsAzure.Storage.Table;

       8:  

       9: namespace ToDoCommon

{

    public class TableTraceListener : TraceListener

    {

        #region Fields      

        readonly string _connectionString;

        readonly string _diagnosticsTable = "CustomLogTable";

  

        [ThreadStatic]

        static StringBuilder _messageBuffer;

  

        readonly object _initializationSection = new object();

        bool _isInitialized;

  

        CloudTableClient _tableStorage;

        readonly object _traceLogAccess = new object();

        readonly List<LogEntry> _traceLog = new List<LogEntry>();

        #endregion

  

        #region Constructors

  

        public TableTraceListener()

            : this("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString")

        {

        }

  

        public TableTraceListener(string connectionString)

            : base("TableTraceListener")

        {

            _connectionString = connectionString;

        }

  

        public TableTraceListener(string connectionString, string tableName)

            : base("TableTraceListener")

        {

            _connectionString = connectionString;

            _diagnosticsTable = tableName;

        }

        #endregion

  

        #region Methods

  

        /// <summary>

        /// Flushes the entries to the storage table

        /// </summary>

        public override void Flush()

        {

            if (!_isInitialized)

            {

                lock (_initializationSection)

                {

                    if (!_isInitialized)

                    {

                        Initialize();

                    }

                }

            }

  

            var context = _tableStorage.GetTableServiceContext();

            context.MergeOption = MergeOption.AppendOnly;

            lock (_traceLogAccess)

            {

                _traceLog.ForEach(entry => context.AddObject(_diagnosticsTable, entry));

                _traceLog.Clear();

            }

  

            if (context.Entities.Count > 0)

            {

                context.BeginSaveChangesWithRetries(SaveChangesOptions.None, 

                    (ar) => context.EndSaveChangesWithRetries(ar), null);

            }

        }

        /// <summary>

        /// Creates the storage table object

        /// </summary>

        private void Initialize()

        {

            var account = CloudStorageAccount.Parse(_connectionString);

            _tableStorage = account.CreateCloudTableClient();

            _tableStorage.GetTableReference(_diagnosticsTable).CreateIfNotExists();

            _isInitialized = true;

        } 

        

        public override bool IsThreadSafe

        {

            get

            {

                return true;

            }

        }

  

        #region Trace and Write Methods

        /// <summary>

        /// Writes the message to a string buffer

        /// </summary>

        /// <param name="message">the Message</param>

        public override void Write(string message)

        {

            if (_messageBuffer == null)

                _messageBuffer = new StringBuilder();

  

            _messageBuffer.Append(message);

        }

  

        /// <summary>

        /// Writes the message with a line breaker to a string buffer

        /// </summary>

        /// <param name="message"></param>

        public override void WriteLine(string message)

        {

            if (_messageBuffer == null)

                _messageBuffer = new StringBuilder();

  

            _messageBuffer.AppendLine(message);

        }

        /// <summary>

        /// Appends the trace information and message

        /// </summary>

        /// <param name="eventCache">the Event Cache</param>

        /// <param name="source">the Source</param>

        /// <param name="eventType">the Event Type</param>

        /// <param name="id">the Id</param>

        /// <param name="message">the Message</param>

        public override void TraceEvent(TraceEventCache eventCache, string source, TraceEventType eventType, int id, string message)

        {

            base.TraceEvent(eventCache, source, eventType, id, message);

            AppendEntry(id, eventType, eventCache);

        }

  

        /// <summary>

        /// Adds the trace information to a collection of LogEntry objects

        /// </summary>

        /// <param name="id">the Id</param>

        /// <param name="eventType">the Event Type</param>

        /// <param name="eventCache">the EventCache</param>

        private void AppendEntry(int id, TraceEventType eventType, TraceEventCache eventCache)

        {

            if (_messageBuffer == null)

                _messageBuffer = new StringBuilder();

  

            var message = _messageBuffer.ToString();

            _messageBuffer.Length = 0;

  

            if (message.EndsWith(Environment.NewLine))

                message = message.Substring(0, message.Length - Environment.NewLine.Length);

  

            if (message.Length == 0)

                return;

  

            var entry = new LogEntry()

            {

                PartitionKey = string.Format("{0:D10}", eventCache.Timestamp >> 30),

                RowKey = string.Format("{0:D19}", eventCache.Timestamp),

                EventTickCount = eventCache.Timestamp,

                Level = (int)eventType,

                EventId = id,

                Pid = eventCache.ProcessId,

                Tid = eventCache.ThreadId,

                Message = message

            };

  

            lock (_traceLogAccess)

                _traceLog.Add(entry);

        }

  

        #endregion

  

        #endregion

    }

}

       1: using Microsoft.WindowsAzure.Storage.Table.DataServices;

       2:  

       3: namespace ToDoCommon

       4: {

       5:     public class LogEntry : TableServiceEntity

       6:     {

       7:         public long EventTickCount { get; set; }

       8:         public int Level { get; set; }

       9:         public int EventId { get; set; }

      10:         public int Pid { get; set; }

      11:         public string Tid { get; set; }

      12:         public string Message { get; set; }

      13:     }

      14: }

What we do here, in summary:

created a storage table

converted captured trace information(trace event cache, type, message, etc.) to a TableServiceEntity object called LogEntry,

then overrided Flush functionality that persists the trace information in table service entity collection to the storage table created in step 1

And now time to add the listener:

       1: public static void ConfigureTraceListener(string diagConn, string tableName)

       2: {

       3:     Trace.Listeners.Clear();

       4:     string traceName = typeof(TableTraceListener).Name; 

       5:     var listener =

       6:         new TableTraceListener(diagConn, tableName)

       7:         {

       8:             Name = traceName

       9:         };

      10:  

      11:     Trace.Listeners.Add(listener);

      12:     Trace.AutoFlush = true;

      13: }

In this code block, we do construct the TableTraceListener with 2 strings:

connection string: set in the service configuration file pointing to the storage table created step 1 above. This is set to “UseDevelopmentStorage=true” when run locally (pointing to local development storage).

table name: the name of the table created in step 1 above.

Logger

Nothing much to say here.

       1: using System;

       2: using System.Text;

       3: using System.Diagnostics;

       4:  

       5: namespace ToDoCommon

       6: {

       7:     public class Logger:ILogger

       8:     {

       9:         static Logger _instance;

        public static ILogger Instance

        {

            get { return _instance ?? (_instance = new Logger()); }

        }

  

        public void LogError(Exception ex)

        {

            var sb = new StringBuilder();

            sb.AppendLine();

            var innerEx = ex.InnerException;

            while (innerEx != null)

            {

                sb.AppendFormat("\t Inner Exc Message: {0}\r\n", innerEx.Message);

                innerEx = innerEx.InnerException;

            }

  

            sb.AppendLine("StackTrace: " + ex.StackTrace);            

            LogEntry(sb.ToString(), EventLogEntryType.Error);

        }

  

        public void LogEntry(string message, EventLogEntryType type)

        {

            switch (type)

            {

                case EventLogEntryType.Error:

                    Trace.TraceError(message);

                    break;

                case EventLogEntryType.Information:

                    Trace.TraceInformation(message);

                    break;

                case EventLogEntryType.Warning:

                    Trace.TraceWarning(message);

                    break;

            }

        }

    }

}

Please note that web role and associated web application run on different processes. Therefore, you need to add the listener in both places.

My Solution

One picture says a million words, so does here:

… and Show Time

All I need to do now, in order to have diagnostic feature enabled in my solution, is to call ToDoCommon.Util.ConfigureTraceListener(diagConnection, tableName) from the method “OnStart” of the role entry point classes of each role (web and worker) and from global.asax. Below, global.asax is shared:

       1: using System;

       2: using System.Web;

       3: using System.Web.Http;

       4: using System.Web.Mvc;

       5: using System.Web.Optimization;

       6: using System.Web.Routing;

       7: using Microsoft.WindowsAzure.ServiceRuntime;

       8: using ToDoCommon;

       9:  

namespace ToDoListWeb

{

    // Note: For instructions on enabling IIS6 or IIS7 classic mode, 

    // visit http://go.microsoft.com/?LinkId=9394801

  

    public class MvcApplication : HttpApplication

    {

        protected void Application_Start()

        {

            AreaRegistration.RegisterAllAreas();

  

            WebApiConfig.Register(GlobalConfiguration.Configuration);

            FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);

            RouteConfig.RegisterRoutes(RouteTable.Routes);

            BundleConfig.RegisterBundles(BundleTable.Bundles);

            AuthConfig.RegisterAuth();

  

            var diagConnection = RoleEnvironment.GetConfigurationSettingValue("DiagConnection");

            var tableName = RoleEnvironment.GetConfigurationSettingValue("DiagTableName");

  

            Util.ConfigureTraceListener(diagConnection, tableName);

        }

  

        protected void Application_OnError(object sender, EventArgs e)

        {

            var ex = Server.GetLastError();

            Server.ClearError();

  

            Logger.Instance.LogError(ex);

        }

    }

}

To test the solution, I have created a dummy page and called the trace functionality from a method:

       1: public ActionResult Details(int id = 0)

       2: { 

       3:     Logger.Instance.LogEntry("Dummy Informative log entry", EventLogEntryType.Information);

       4:     Logger.Instance.LogEntry("Dummy Warning log entry", EventLogEntryType.Warning);

       5:     Logger.Instance.LogEntry("Dummy Error log entry", EventLogEntryType.Error);

       6:  

       7:     var todoitem = _context.ToDoItems.Find(id);

       8:     if (todoitem == null)

       9:     {

      10:         throw new ArgumentOutOfRangeException();

      11:         //return HttpNotFound();

      12:     }

      13:     Trace.Flush();

      14:     return View(todoitem);

      15: }

And the result confirms the success!

Conclusion

In this post, I have implemented a custom trace listener that captures the diagnostic events (trace, debug, and trace source) in the application and writes to a Windows Azure storage table. Advantages of this custom tracing are being able to implement the instrumentation the way you want (lighter perhaps) and being able to define the table name yourself.

Please note that the code blocks shared here are for demonstration purposes; I would suggest you to review your needs (trace switch, buffering, application monitoring capability, etc.) before using them in alive systems. Additionally, I do recommend investigate the configure before customization approach which I will be talking in my next post. Stay tuned

Install Active Directory in Windows Azure in four simple steps

Meged Ezzat — Tue, 21 May 2013 10:49:42 GMT

Introduction:

There is a very long and complex way to install Active Directory in Windows Azure environment described in details in Windows Azure documentation here http://www.windowsazure.com/en-us/manage/services/networking/active-directory-forest/, what I will propose in this post is a simple way to install Active Directory in a new Windows Azure without the need to go through all steps listed in Microsoft Windows Azure documentations that include creation of the virtual machine for Domain Controller using PowerShell.

Install Steps:

Installation steps are simple as follows:

1. Create a storage account that is in the same region as the affinity group, more details about how to create a storage in Windows Azure can be found here http://www.windowsazure.com/en-us/manage/services/storage/how-to-create-a-storage-account/.

2. Create a new Network associated with the storage account created in step-1, create a Network subnet (for example 172.16.0.0/24), and select DNS to be same Network VLAN IP Address end with “.4” (follow the same example 172.16.0.4), more details about how to create a virtual network in Windows Azure can be found here http://www.windowsazure.com/en-us/manage/services/networking/create-a-virtual-network/.

3. Create the first virtual machine and associate it with the storage account and network created in steps-1 & step-2, this virtual machine automatically will take IP Address from network VLAN end with “.4” (follow the same example 172.16.0.4), more details about how to create a virtual machine in Windows Azure can be found here http://www.windowsazure.com/en-us/manage/windows/tutorials/virtual-machine-from-gallery/.

4. Install Active Directory (Including DNS Service) on the first created virtual machine in step-3.

Now your Windows Azure environment is ready with Active Directory and all new virtual machines that will be created later will automatically point to the IP Address of the Active Directory Domain Controller IP Address as their primary DNS server in network cards.

Conclusion:

As a conclusion, you have the long professional way to install Active Directory, and in this post, you get how to install Active Directory in your new Windows Azure environment with simple way and very fast.

Hope that this will accelerate building your new Windows Azure environment in less than an hour.

Is AD integration or manual SCOM agents installation required to allow Automatic Windows Agent Failover between multiple SCOM 2012 Management Servers?

Meged Ezzat — Fri, 10 May 2013 14:34:00 GMT

Introduction:

I observed that some IT professionals confused about System Center Operations Manager (SCOM) 2012 windows agents failover requirements, and the raised question was do I need to configure SCOM Active Directory Integration or do manual agents implementation or use PowerShell to allow windows agents to do automatic failover between multiple SCOM management servers? or should I wait and delay SCOM agents deployment until deploying System Center Configuration Manager to deploy SCOM agents? and the simple answer is “No” need, and below you can find more details about from where confusion come and why you do not have to wait until AD integration or Configuration Manager be implemented to push SCOM windows agents, and how to verify that SCOM windows agents failover working properly.

Detailed Description:

First, confusion come because some documentation and blog posts shows how to distribute windows agents through SCOM management servers using AD integration or PowerShell Scripts to automate agents assignment through different SCOM management servers which is very good, however this is not mandatory specially for small environment that includes less than 100 windows agents, and the reason behind is that you can distribute the windows SCOM agents though the SCOM management servers while pushing the agents from SCOM management console, so simply you can select which management server will be the primary management server for agents that you are going to push right now as shown in below diagram that you can scroll down and select which management server will be the primary management server for agents that will be implemented within this wizard:

So the simplest way for small to medium windows servers environment is to push SCOM agents between the available management servers in the same management servers pool, so for example if you have two management servers automatically will be member in the same management servers pool then you can push 50% of agents to have SCOM management server-1 as their primary management server and then push the other 50% of agents to have SCOM management server-2 as their primary management server.

To verify windows agents failover between the different management servers simply you need to stop one of the management servers (disconnect from network or shutdown the management server) and check the Operations Manager event viewer in one of the managed computers by the stopped SCOM management server so you will see event ID 21006 as shown below telling you that there is a problem to connect to his management server.

If heart beat configuration kept with default value which is 60 seconds then another event ID 21015 will be generated as shown below telling you that OpsMgr was unable to set up a communications channel to the agent primary management server.

3 Seconds later you will see a new event ID 21017 will be generated as shown below telling you that OpsMgr has successfully failed over to the secondary management server

The good point here is that when the windows agent primary server come back running agent will fail back automatically to its primary management server.

Conclusion:

As a conclusion, SCOM windows agents automatic failover does not require AD integration or PowerShell scripting or Configuration Manager or manual agents installation specially for small to medium environment and agents distribution between different SCOM management servers can be accomplished through push agents wizard, and windows agents failover can be simply verified from event viewer.

Office 365 Arabization – Part IV

Mohamed Marzouk [MSFT] — Fri, 10 May 2013 08:38:30 GMT

In previous parts I’ve explained users’ bulk creation (Part II) and bulk modification (Part III).

In this part let’s talk about more advanced management tasks.

Dynamic Distribution Groups:

Dynamic Distribution Group (DDG) is special group created based on conditions not based on list of members as the normal group. Membership of the DDG is calculated each time an email is sent to the DDG, the calculation is based on the filters and conditions specified during the creation of the DDG.

There are two types of the DDG:

- Precanned filters: this the easy DDG where we can use the following filters:

o Recipient type.

o Company.

o Custom attribute.

o Department

o State.

- Customized filter: where we can use any attribute

Before starting with creating the DDGs, our challenge here is not creating DDG, as the title of these series it’s about Arabization. So the challenge is to create Arabic DDG with Arabic conditions.

You will need to review Part I which discussing how to use Windows PowerShell ISE to solve the Arabic problem.

Once you have ISE installed we will need to connect to the tenant and remember DDG is for Emails so yes this will need connecting to Exchange Online:

1. Connect to the tenant (AD): Connect-MsolService

2. Connect to Exchange Online:

$cred=Get-Credential

$session=New-PSSession -ConfigurationName microsoft.exchange -ConnectionUri:https://ps.outlook.com/powershell -Credential:$cred -Authentication basic -AllowRedirection

Import-PSSession $session

Now we are connected to Exchange Online and you can use Exchange cmdlets

Let’s check the DDG examples:

Precanned Filters:

Target is: create DDG with only users live in "التجمع الخامس".

From the above explanation in Precanned filters DDG we can filter by State (not city), so as you see in the following snapshot, we have some users in "التجمع الخامس", some users in "الهرم", and other users State value is empty.

The challenge is when using the following cmdlet with only the mandatory attributes:

New-DynamicDistributionGroup -Name التجمع الخامس"" -IncludedRecipients mailboxusers -ConditionalStateOrProvince "التجمع الخامس”

Check the following result of Get-DynamicDistributionGroup |fl

In the above snapshot, check the marked values:

- Alias

- EmaildAddresses

- PrimarySmtpAddress

As you can see the values are Arabic and driven from the name specified in the first cmdlet "التجمع الخامس" and of course this DDG will not work, because these values are only supported in English (until that day when we see my email محمد@مرزوق.مصر).

So what we need is to create the DDG using the following cmdlet:

New-DynamicDistributionGroup -Name "التجمع الخامس" -IncludedRecipients mailboxusers -ConditionalStateOrProvince "التجمع الخامس" -Alias Tagamo3 -PrimarySmtpAddress tagamo3@meamcs.com

Then check the properties of the DDG:

Now these attributes in English. However the display name is still in Arabic, so the users will see the DDG in Arabic.

Now let’s move to another topic, how are we going to make sure that this DDG is really working and the users in التجمع الخامس are members of this group.

Remember this is not a static membership, so what we need to do is to calculate the members of this DDG:

$ddg=Get-DynamicDistributionGroup "التجمع الخامس"

Get-Recipient -RecipientPreviewFilter $ddg.RecipientFilter

Customized Filters:

This is more advanced where we can use any attribute and use AND, OR.

The target is to create DDG for users with mailbox and with title “استشاري”.

Let’s first check our users:

As explained before we will need to specify Alias, PrimarySMTPAddress values to get healthy DDG:

New-DynamicDistributionGroup -Name استشاري -Alias isteshary -PrimarySmtpAddress isteshary@meamcs.com -RecipientFilter {(recipienttype -eq "usermailbox") -and (title -like "استشاري")}

In the above cmdlet the filter we have used is recipienttype was equal to usermailbox and this mailbox user must also has titleاستشاري .

Let’s do final check to see the members of this DDG:

In this part I’ve explained the Dynamic Distribution Groups, next part we will talk about some reporting and filtering cmdlets.

Part I: Arabic Problems

Part II: Users Bulk Creation

Part III: Users Bulk Modification

Exchange 2013 Coexistence ECP “Your mailbox can't be accessed using the address you entered, please obtain the correct address”

Tamer Sherif Mahmoud — Sat, 04 May 2013 13:46:00 GMT

In a coexistence scenario, where you’re running Exchange 2007 or Exchange 2010 and Exchange 2013 in the same organization, and your mailbox is still hosted on the Exchange 2007/2010 Mailbox server and you try to access the Exchange 2013 default /ECP directory, you will get the following error:

“Your mailbox can't be accessed using the address you entered, please obtain the correct address, please contact your helpdesk”

Method1:

To proceed to the control panel you can access the EAC by adding the Exchange version to the URL. For example, to access the EAC whose virtual directory is hosted on the Client Access server locally, use the following URL: https://localhost/ecp?ExchClientVer=15.

Same way applies if you want to access the Exchange 2010 ECP and your mailbox hosted on an Exchange 2013 Mailbox server, use the following URL: https://localhost/ecp?ExchClientVer=14

Method2:

Through PowerShell create a new mailbox on Exchange 2013 and give it the necessary administrative permissions.

Securing Dynamic Data ASP.NET SQL Azure Published Web Site with ACS and Facebook as an Identity Provider

Mohamed M Malek — Sat, 27 Apr 2013 15:50:24 GMT

The Scenario

I wanted to implement an Azure web site that is using the Azure Access Control Service and integrates with an external identity provider to authenticate and authorize users. At first I thought of using Windows Live ID but it has a problem that the only claim offered by WLID is the unique identified which is simply a number and represents nothing from the user. Then I thought why not make things more interesting and use Facebook. I think things got more interesting than I thought. J

I wanted to implement a Dynamic database access web site so that it generates the views on top of an existing SQL Azure database and lets the end user manipulate the database tables and filter them. This is using Linq-to-SQL classes.

I am using Visual Studio 2012 latest version.

The Steps highlights

The steps at a glance of how to get this up and going are as below:

1- Create your project:

a. Create your database.

b. Create a new Dynamic ASP.NET project.

c. Add a Linq-To-SQL model to your database.

d. Change the Framework version of the project.

e. Set the “ScaffoldAllTables” to true.

2- Download the latest “Identity and access tool”

3- Create a new Azure web site and download the publishing settings.

4- Setup your identify provider:

a. Create a new Azure ACS namespace

b. Create a new Facebook application

c. Configure your Facebook application.

d. Add your Facebook application as an identity provider.

e. Configure you claims rules

5- Set the ACS settings in the “Identity and access tool”

6- Implement your custom claims authorization manager

7- Complete the web.config configuration

8- Publish your web site.

Solution Walkthrough and description

Create your project

This is the first step and as I described I wanted to create a dynamic ASP.NET site based on a custom database.

Create the Database

First I created the database in SQL Azure.

Created a new SQL server and provided the administrator permissions and allowed Azure services access to this server.

Then I allowed access to my IP to this database to allow me to manage it.

Then I started to design my database (this can be either done online or using Visual Studio)

Or from Visual Studio 2012

Once the database is created and ready to be used then you will go to the next step.

Create the Dynamic ASP.NET project

Open visual Studio and click on new project. You will need to switch to .net framework 4.0 to see the dynamic data ASP.NET SQL template.

Change the “Global.ascx” file to uncomment the line to connect to the classes to be as follows

DefaultModel.RegisterContext(typeof(TestDbDataClassesDataContext), new ContextConfiguration() { ScaffoldAllTables = true });

Change the .net Framework version to be 4.5 to be able to see the Identity and Access tool link.

Download the Identity and Access tool

From the visual studio tools menu click on the “Extensions and updates”

Search for the “Identity and Access tool” and install it and restart your Visual Studio.

Create the Azure Web Site

Open the Azure management site and click on new web site.

Click on “Download the publish profile”

Save this file on your Hard disk. Now in Visual Studio click on the project and then Publish

Click on “Import”

Now select the publish settings file already downloaded.

Complete the publishing and test that the application is now published and working.

Now the next step is to configure the Facebook as an identity provider.

Setup Facebook as an Identity Provider

Create the Facebook application

Logon to your Facebook account and then open the link http://developers.facebook.com

Now click on Apps and then create a new App

Give your application a name

Take note of your application ID and secret.

Also enter the URL of the ACS namespace you will create on the Azure ACS web site in the next step (It is better to create that namespace first and then return to this step later).

Click save Changes.

Create Your ACS Namespace

Open the Azure portal and click new to create a new ACS names space.

Once created you can click on the Manage link to start managing it.

Configure your ACS service

Start by adding the Facebook application as an identity provider. Click on identity providers and then “Add”

Enter the application ID and secret and click “Save”

Configure Your Project to Link ACS Service

While on the ACS management site click on “Management service” so see all management links required to be able to communicate with ACS.

Click on “Show Key” and then copy the symmetric key generated.

These will be the namespace and the management key of the namespace.

Right click on your project and then click “Identity and Access tool”

Configure your ACS with the namespace and the key already copied before after selecting “Use the Windows Azure Access Control Service”.

Now select the settings as below

Finally click “Ok”. This will configure your ACS service and the relying party application with the required pass-through rule for all provider claims as shown below.

The next steps are to create and implement the Claims authorization manager and configure your web.config.

Implement a Custom Claims Authorization Manager

Since we are using .Net framework 4.5 this is a little different than what we used to do in 3.5 since now the WIF is totally integrated in the framework.

Add reference to the System.IdentityModel assembly

Add and implement the new class “MyAuthorizationManager”

This is done so that the code of the file would be as follows.

using System.IO;

using System.Xml;

using System.Collections.Generic;

using System;

using System.Web;

using System.Linq;

using System.Security.Claims;

namespace TestDbWebApplication

{

public class MyAuthorizationManager : ClaimsAuthorizationManager

{

private static Dictionary<string, string> m_policies = new Dictionary<string, string>();

public MyAuthorizationManager()

{

}

public override void LoadCustomConfiguration(XmlNodeList nodes)

{

foreach (XmlNode node in nodes)

{

//FIND ZIP CLAIM IN THE POLICY IN WEB.CONFIG AND GET ITS VALUE

//ADD THE VALUE TO MODULE SCOPE m_policies

XmlTextReader reader = new XmlTextReader(new StringReader(node.OuterXml));

reader.MoveToContent();

string resource = reader.GetAttribute("resource");

reader.Read();

string claimType = reader.GetAttribute("claimType");

if (claimType.CompareTo(ClaimTypes.Name) != 0)

{

throw new ArgumentNullException("Name Authorization is not specified in policy in web.config");

}

string name = "";

name = reader.GetAttribute("Name");

m_policies[resource] = name;

}

public override bool CheckAccess(AuthorizationContext context)

{

//GET THE IDENTITY

//COMPARE WITH THE POLICY

string allowednames = "";

string requestingname = "";

Uri webPage = new Uri(context.Resource[0].Value);

ClaimsPrincipal principal = (ClaimsPrincipal)HttpContext.Current.User;

if (principal == null)

{

throw new InvalidOperationException("Principal is not populate in the context - check configuration");

}

ClaimsIdentity identity = (ClaimsIdentity)principal.Identity;

if (m_policies.ContainsKey(webPage.PathAndQuery))

{

allowednames = m_policies[webPage.PathAndQuery];

requestingname = ((from c in identity.Claims

where c.Type == ClaimTypes.Name

select c.Value).FirstOrDefault());

}

else if (m_policies.ContainsKey("*"))

{

allowednames = m_policies["*"];

requestingname = ((from c in identity.Claims

where c.Type == ClaimTypes.Name

select c.Value).FirstOrDefault());

}

if (allowednames.ToLower().Contains(requestingname.ToLower()))

{

return true;

}

return false;

}

This would authorize users based on their login name reported by the identity provider (Facebook).

Configure the Authorization manager in the Web.Config

The required steps are to add the following lines:

<system.webServer>

</modules>

</system.webServer>

Please note the yellow highlighted section above as this is key to make this work.

<system.identityModel>

</securityTokenHandlers>

</identityConfiguration>

</system.identityModel>

Please note the highlighted section as publishing to Azure web site will not work without these lines.

Finally the claims authorization manager configuration.

<system.identityModel>

</policy>

</claimsAuthorizationManager>

</identityConfiguration>

</system.identityModel>

Publish and test your site

Now you have completed the site configuration publish it and you should be able to authenticate using Facebook and authorize the application to use your Facebook settings, and finally authorize the user.

Logon to Facebook as usual.

The app would request user permissions to give the user details to the ACS service.

Web site will work as required.

Happy coding J

Windows Azure – Remove Virtual Disks

Mohamed Marzouk [MSFT] — Fri, 26 Apr 2013 10:43:11 GMT

This is very famous issue when you create many Virtual Machines on Windows Azure, after deleting the VMs the virtual disks (VHD files) are still there.

So far there is no way to remove these VHD files using the GUI interface, so I started to search how can I do it through PowerShell. The challenge of Azure PowerShell is that most of the resources available for developers using Visual Studio, for me any thing starts with Visual Studio is sort of mystery (The nice story that one of my developer friends identify SharePoint Site Collection: is a site contains collections is true story) that should be left to wisdom men (like this friend) who can understand it .

For Infrastructure guys like me, the following are step by step of how to use PowerShell to connect to Azure and then delete the VHDs:

Step 1: Management Certificate:

In simple words we will need a certificate to connect with PowerShell to Azure, this certificate should be 2048 bits at least, self signed certificate is good option for testing and labs.

Creating Self Signed Certificate can be done by many ways, the easiest way is using the normal PowerShell:

1. Open PowerShell (as an Administrator).

2. Type the following cmdlet:

New-SelfSignedCertificate -DnsName azure -CertStoreLocation cert:\LocalMachine\My

This cmdlet will create new self signed certificate and place it in the local machine store, check the following snapshots:

Let’s make sure that the certificate is created:

1. From Run > Type mmc.

2. Click File > Add/Remove Snap-in.

3. Select Certificate and Click Add.

4. Select Computer Account > Finish.

5. Under Personal > Certificate > you should find the new certificate “azure”:

The next step is to upload this certificate with private key (the PFX file) to Windows Azure, in order to do that you will need to export the PFX file (the certificate with the Private Key) first:

1. From the same Snap-in.

2. Right click the certificate > All Tasks > Export.

3. Click Next > Select “Yes Export the Private Key” > Click Next.

4. On Export File Format click next > Then provide the password.

5. Browse to the location where you will save the certificate.

From Azure Management Portal create a new Cloud Service to use it with the certificate:

After having the PFX file we will need to upload it to Windows Azure:

Log into the Management Portal.
In the navigation pane, click Cloud Services, and then click the service for which you want to add a new certificate.
On the ribbon, click Certificates, and then click Upload a Certificate.
In the Upload certificate dialog, click Browse For File, go to the directory containing your certificate, and select the .pfx file to upload.
Type the password of the private key for the certificate.
Click OK.

Step 2: Install Azure PowerShell

To install Windows Azure PowerShell:

Download Windows Azure PowerShell: http://go.microsoft.com/?linkid=9811175&clcid=0x409
Install Windows Azure PowerShell.

Step 3: Connect to Azure:

Before we start using the Windows Azure cmdlets to remove the VHDs or anything else, we will need to configure connectivity between the workstation and Windows Azure. This can be done by downloading the PublishSettings file (this file will setup the PowerShell environment to use Windows Azure) from Windows Azure and importing it. The settings for Windows Azure PowerShell are stored in: <user>\AppData\Roaming\Windows Azure PowerShell.

From Windows Azure PowerShell type the following cmdlet:
Get-AzurePublishSettingsFile

A browser window opens at https://windows.azure.com/download/publishprofile.aspx, where you can sign in to Windows Azure.
Sign in to the Windows Azure Management Portal, and then follow the instructions to download your Windows Azure publishing settings. Use your browser to save the file as a .publishsettings file to your local computer. Note the location of the file.
In the Windows Azure PowerShell window, run the following command:
Import-AzurePublishSettingsFile FileName.publishsettings

Step 4: Remove the VHDs (Finally)

Once we are connected now to Azure, we can manage Azure with PowerShell and do anything we need, if you forgot what was the purpose of this article this is the time you will need to scroll up and read it again

Run the following cmdlet to see all the virtual disks (the VHD file):

Get-Azuredisk

In the above snapshot this is the disk that I want to remove (you will need to make sure that is not connected to any Virtual Machine), then run the following cmdlet:

Remove-AzureDisk <DiskName>

Now you will probably decide to leave the VHDs and not delete them which is easier , I don’t blame you for that. But you will have this article as reference in case one day someone asked you the question: How can I remove the unused VHD file.

for more details about Azure PowerShell: http://msdn.microsoft.com/en-us/library/windowsazure/jj156055.aspx

for more information about Azure Management Certificate: http://msdn.microsoft.com/en-us/library/windowsazure/gg981929.aspx

Office 365 Arabization – Part III

Mohamed Marzouk [MSFT] — Wed, 24 Apr 2013 08:07:28 GMT

In Part II I’ve explained how to create bulk users using CSV file, and how to create the CSV file.

In this part we will discuss some of the management tasks including password.

Users Bulk Import with Password

In Part II I’ve explained how to prepare the CSV, create the users with randomly generated password, which will require sending this password later to the users, check the following report after creating the users:

In large environments it may be required to create users with password (could be generated from ID as an example) and of course enforce the user to change it once logon, check the following cmdlet:

Import-Csv D:\arab-users.csv | ForEach-Object -Process {New-MsolUser -UserPrincipalName $_.upn -FirstName $_.firstname -LastName $_.lastname -Department $_.department -Title $_.title -City $_.city -Country $_.country -DisplayName $_.displayname -LicenseAssignment meamcs:ENTERPRISEPACK -UsageLocation EG -Password $_.password -ForceChangePassword $true} | Export-Csv -Path c:\arabusers-result.csv -Encoding unicode

In the above cmdlet we have completed the following:

1. Create the users in the CSV file with the mentioned attributes.

2. Assign license with usage location (which is required).

3. Set each user’s password according to the value in the CSV file.

4. The user will be forced to change the password on the next logon ($true is the default value).

5. Export the result in CSV file.

Let’s first discuss the second point how I got LicenseAssinment value, run the following cmdlet to get the license in your tenant:

Last thing is to make sure that the password will be according to the below conditions:

- Use 8 to 16 characters.

- Create a strong password. Office 365 requires at least 3 of the following:

o Lowercase characters

o Uppercase characters

o Numbers (0-9)

o Symbols, including: ! @ # $ % ^ & * - _ + = [ ] { } | \ : ‘ , . ? / ` ~ “ < > ( ) ;

- Don’t use the user name (the part of your user ID to the left of the @ symbol).

- UNICODE values are not allowed, so no Arabic password J

Users Bulk Modify

After creating users it may be required to modify users’ attributes, so we will need to have our CSV file ready as per the Part II, then the following cmdlet shows an example of modifying some attributes:

Import-Csv D:\arab-users.csv | ForEach-Object -Process {Set-MsolUser -UserPrincipalName $_.upn -FirstName $_.firstname -LastName $_.lastname -Department $_.department -Title $_.title -City $_.city -Country $_.country -DisplayName $_.displayname -UsageLocation EG -PasswordNeverExpires $true} | Export-Csv -Path d:\arabusers-result.csv -Encoding unicode

Note:

1. From the CSV and for each user with this UserPrincipalName value ($_.upn) we have modified all of the above attributes to the values in the CSV file.

2. UsageLocation value has been used with the same cmdlet.

3. Password Never Expire has been allowed, so the password will not expire (the default is the password will expire every 90 days).

The following can’t be done from this cmdlet:

1. Change the User Principal Name, which can be done through the following cmdlet:

Import-Csv C:\ newupn.csv | ForEach-Object -Process {Set-MsolUserPrincipalName -UserPrincipalName $_.upn -NewUserPrincipalName $_.upn1}

2. Resetting the password, which can be done by the following cmdlet:

Import-Csv d:\arab-userspassword.csv | ForEach-Object -Process {Set-MsolUserPassword -UserPrincipalName $_.upn -NewPassword $_.password}

3. Assign license to the users, and this can be done by using the following cmdlet:

Import-Csv d:\arab-userslicense.csv | ForEach-Object -Process {Set-MsolUserLicense -UserPrincipalName $_.upn -AddLicenses "meamcs:ENTERPRISEPACK"

Last nice trick to modify everything in one cmdlet by pipeline all the above cmdlets as the following:

Import-Csv -Path C:\arabusers.csv | ForEach-Object -Process {Set-MsolUser -UserPrincipalName $_.upn -UsageLocation EG | Set-MsolUserLicense -UserPrincipalName $_.upn -AddLicenses "meamcs:ENTERPRISEPACK" | Set-MsolUserPassword -upn $_.userprincipalname -NewPassword $_.password}

And don’t forget the “” with the license.

I hope the above cmdlets were useful and saved time of trying and searching how to do it. In the next part I will explain the reports and Dynamic Distribution Groups.

Review the previous parts:

Part I

Part II

Some users unable to create VMs in VMM 2012 SP1: “User or user role not valid” (Error ID 26726)

Fatima Azzahra El Azzouzi — Tue, 23 Apr 2013 12:48:00 GMT

This post describes an error that occurred during System Center Virtual Machine Manager 2012 SP1 deployment, with two VMM servers in cluster configuration.

When using the VMM Console, some users (but not all users) were unable to create VMs, even if the user account is a member of the Delegated Administrator role. In this case, when the user selects “Create a new VM”, the following error message is displayed when “Next” is clicked on the “Configure Hardware” page:

ID 26726: “Either the specified user role or the specified user (%Username) is not valid. User is not a member of the role. Add (%Username) as a member of the user role and try again or provide a different user role or a different user.”

The same error persists even if the user accounts become VMM Administrators.

This error originates from a known issue (http://support.microsoft.com/kb/331951) where the VMM service does not have access to authorization information on user account objects or computer account objects. Specifically, the VMM service cannot read the token-groups-global-and-universal (TGGAU) attribute in AD.

This issue is resolved by adding the VMM Service account to the Windows Authorization Access (Pre-Windows 2000 Compatible Access) group in AD.

In conclusion, if some users are unable to create VMs through the VMM Console due to Error ID 26726, the VMM service is probably unable to verify whether those users are authorized to create VMs, and adding the VMM service to the Pre-Windows 2000 Compatible Access group resolves the issue.