Bobby and Nima's Forefront Identity Manager Blog

Run on Policy Update - Retroactive Policy Enforcement

Somewhat hidden within Forefront Identity Manager 2010, there is a very useful feature for action workflows called "Run on Policy Update". 

Here are the situations where you may find this feature useful:

1.  You are creating a new Management Policy Rule (MPR), such as one to provision all users an AD account, and you want one or more of the action workflows in your new MPR to be applied, upon creation of the MPR, to all the members of the MPR's Resource Final Set (also referred to as "Target Resource Definition After Request" in the portal's MPR wizard).  For example, you may be creating a new MPR to apply a new Synchronization Rule to all users.  You may want to retroactively enforce this new policy by applying the Synchronization Rule workflow to all users that already exist.

2. You are enabling a previously disabled MPR, and you want one or more of the action workflows in the MPR to be applied, upon enabling of the MPR, to all the members of the MPR's Resource Final Set.

3. You are adding a new action workflow to an existing MPR, and you want the new workflow to be applied to all the members of the MPR's Resource Final Set, immediately upon adding the workflow to the MPR.

4. You are modifying the Resource Final Set of an existing MPR to reference a new set, and you want one or more of the MPR's action workflows to be applied to all the members of the new Resource Final Set, immediately upon modification of the MPR.

5. You are manually modifying the membership of the Resource Final Set of an MPR, either by modifying the set's Filter or ExplictMember attribute, and you want one or more of the MPR's action workflows to be applied to all the *new* members of the new Resource Final Set, immediately upon modification of the set.

The "Run on Policy Update" feature is an option that lives on action workflow definitions, as an attribute labeled "RunOnPolicyUpdate" bound to the WorkflowDefinition resource type.  When this boolean attribute is set to "true" for a given action workflow, if any of the 5 scenarios above are encountered with an MPR that uses this workflow, the workflow will be automatically applied to the members of the Resource Final Set of the MPR.

Following is a table that summarizes the cases where a "Run on Policy Update" enabled action workflow is applied, in addition to the normal cases where a new Request satisfies all the criteria of an MPR that uses the workflow.

Nima

Farewell!

As a lot of you may already know, after 4 years of working on MIIS / ILM / FIM I've decided to leave Microsoft. No, I'm not being fired, nor am I jumping ship to a competitor :) Rather I am leaving Microsoft to pursue a MBA at Columbia starting this fall.

 If I have one hope is it that you have found this blog, along with the talks, webcasts and reports I have done to be useful in helping to digest the seemingly endless new concepts coming out with FIM 2010. It can be a lot to take in one shot for sure, as sometimes I even find myself scratching my head as to how something works and even more so when I attempt to describe it,

 To all of my colleagues and friends I have made in my time on the MIIS/ILM/FIM teams, I wish you all the best going forward. As for me, its back to the student life for the time being.

 Going forward, I'll still be available to help people with FIM (as much as my new student life permits of course!) if you want to reach me, you can email me at bobby.gill (at) gmail.com and you can always find me on facebook at www.facebook.com/jasjeet

 Ill be at TechReady tomorrow morning and I should be at the FOX Sports Grill event in the evening, so come grab me if you see me.

 In the meantime, I leave this blog in the more than capable hands of my colleague Nima.

 Thanks

Bobby

Opening the mailbox:

 Question from one of our readers:

From:
Sent: Tuesday, July 14, 2009 11:06 AM
To: Bobby Gill
Subject: (Bobby and Nima's Forefront Identity Manager Blog) : Question about FIM/Outlook
Importance: High

Good afternoon Bobby and Nima's.

I read your blog often as I await the release of MMS...I mean MIIS...I mean ILM...I mean FIM :)

I keep seeing things about how nice it is to request to be added to groups or distribution groups through Microsoft outlook. Are all of the features available from inside Outlook available from the web interface?

The reason I'm asking is that a lot of clients may have a need to use identity management to synchronize their Microsoft world with their HR world...and their email world ! (*cough Lotus Notes*). I understand that the level of integration (especially for distribution lists) is probably not the same, but are most other features of the Outlook FIM client available on the web console ?

------

The simple answer to the question above is that the Outlook plug in for FIM 2010 contains a proper subset of functionality available within the web portal. That is everything that is possible through the Outlook interface is available within the web portal.

However the opposite is not true. While the Outlook plug in allows you to manage group memberships and approvals/requests, it does not nearly provide the same level of functionality as the web portal. For instance the creation of groups (both static and dynamic), deletion and modification (outside of membership) can only be done through the FIM portal.

Further, the Outlook plugin requires both Outlook 2007 as well as Exchange 2007 running on the backend. However, if you are using an email client which is not Outlook 2007, or a email server that is not Exchange 2007, you can still send notifications and approvals to email clients via any SMTP server. To perform any operations on said messages will require you to go to the portal and perform .

Introducing Hierarchical Provisioning

Yesterday I was eating my bowl of Frosted Miniwheats (by Kellogg) for dinner  and out fell a coupon for another free box of Miniwheats. “Oh hot lam!” I exclaimed to myself. I had totally not seen the offer stamped on the front of the box for a free box of Miniwheats when I had purchased the jumbo, bachelor sized box of Miniwheats at Costco last Sunday.  A bowl of Miniwheats alone is enough to brighten my days, but winning another 12oz of the half-sugar, half-fiber narcotic ? Well that’s like Christmas in May. I love hidden surprises.

Much like my box of Frosted Miniwheats, FIM 2010 has a few hidden surprises of it’s own that lurk underneath the covers and are often ignored. One of these features is Hierarchical Provisioning. Much like the name would imply, Hierarchical Provisioning allows objects, and more importantly, any missing parent containers, to be provisioned into the connector spaces of LDAP MAs . Previously in MMS, MIIS, and ILM 2007, if one wanted to provision a user into a container in Active Directory, one would need to ensure that they created the container in Active Directory prior to provisioning the user with MMS/MIIS/ILM. However, with Hierarchical Provisioning, you do not need to do this anymore. With some settings configured in the Management Agent (MA), the missing container can be created automatically by the Active Directory Management Agent, and then the object provisioned within it.

The steps to configure this feature are relatively straight forward. Assume that you want to provision the following user into Active Directory: “cn=Bobby Gill, ou=Redmond, ou=Users, dc=fabrikam, dc=com”. In this case, the Redmond OU does not exist in the Active Directory domain. Before the ILM AD MA can provision this new user into the OU specified,  the OU needs to be created in Active Directory. This is where Hierarchical Provisioning comes into play.

 As an ILM Admin, to enable Hierarchical Provisioning on a LDAP MA, you need to configure a mapping within the MA such that anytime upon export the MA detects that a parent of a object doesn’t exist, it knows what object to create in the connected directory for that parent. This configuration is done within  the LDAP MA screens  by mapping valid DN components to object classes in the connected directory. In this case, you would set up a mapping between the “OU” DN component to the object class “organizationalUnit”. Thus in the above scenario, when the MA is exporting the object to AD and realizes that the “OU=Redmond” parent is missing, it will look up the mapping for the “OU” component and first create a new organizationalUnit object named “Redmond” and then export the new user  into the container.

 Steps to configure Hierarchical Provisioning: 

1. Create a new instance of your favorite LDAP Management Agent. Personally, I’m a Microsoft guy, so obviously I always choose Active Directory Domain Services.
2. You’ll notice a new page on the left tab titled ‘Configure Provisioning Hierarchy’.
3. Map DN components to Object Classes. The DN Component list box lists all known valid DN components for the given directory, this is inferred by analyzing the LDAP schema of the directory. To the right is the list of available object classes in the directory, again taken from the LDAP schema.

4.)    Mappings are created by selecting a DN component in the left list box, and a object class in the right list box, and then clicking “new”. You can only create 1 mapping per DN component.

 Once setup, Hierarchical Provisioning is transparent to the actual provisioning mechanism. Thus, if you are using Synchronization Rules or even a traditional scripted Metaverse Extension, these settings will be applied to both at export time.  Hierarchical Provisioning further reduces the burden on IT Pros by allowing much more flexibility in terms of provisioning decisions made in the FIM Workflows and eliminates an often tedious manual step whenever a new business unit comes online and an associated container or OU needs to be created.

The feature is available to all LDAP Management Agents and is available in the ILM "2" RC0.

Topics Wanted!

The hardest part of this blog is finding topics to write about that would be interesting and useful to the community at large. If you have a topic or a question that you want to see addressed on this blog, please email me and I will see if I can post something up for it.

Dependencies - not just for avoiding taxes!

What's our name again?

Whoa, new product name! For those of you who have been chasing butterflies for the past month, what was once known to us as Identity Lifecycle Manager "2" is now called Forefront Identity Manager.  I know, it's not the sexiest name in the world and is probably the 5th different name the product has had since it's conception, but it reflects the combination of Microsoft's security and identity product lines into the Forefront brand announced last year.

Personally, I wanted to name the product "Black Thunder II", but then again there are a myriad of reasons why I am not allowed to name Microsoft products.

 But back onto the topic at hand...

Synchronization Rule Dependency

I decided to take some time off today to briefly talk about Synchronization Rule Dependencies, a powerful yet not well understood part of ILM FIM's synchronization capabilities. In brief, a Synchronization Rule Dependency allows one to construct and apply a series of outbound Synchronization Rules ontop of each other. The scenarios that spring to mind whereupon this functionality is useful are things such as adding/removing Exchange mailbox provisioning, or adding/removing VPN access upon a user's Active Directory account (with the former 2 being dependent on the latter).

If an Outbound Synchronization Rule (the dependent) is marked as having a dependency on another Synchronization Rule (the root), the dependent rule will apply itself ontop of the connector that the root Synchronzation Rule is applied on. At run time, when a FIM Action Workflow attempts to add an Expected Rule Entry (ERE) object for the dependent Synchronization Rule onto a FIM Resource's Expected Rules List (ERL) , there needs to also exist an ERE-Add object for the root Synchronization Rule on the ERL.  (I am just going to take a minute here and say I don't think there has been that many acronyms stuffed into one sentence since the merger between the wrestling giants WWF and WCW was announced). Conversely, if an Action Workflow adds a ERE-Remove entry for a root Synchronization Rule, all EREs that correspond to Synchronization Rules further up the dependency tree will be removed.

Its important to note that when you design an Action Workflow to add or remove a series of EREs that correspond to a Synchronization Rule dependency chain, the root rule must be added to the workflow surface prior to any other dependent rules.

Multiple levels of dependency can be created, with more than one Synchronization Rule being made to depend on a single Synchronization Rule.

In the Synchronization Rule Designer, to create a Synchronization Rule Dependency is relatively straightforward. The first page of the designer allows you to select another outbound Synchronization Rule to make a new Synchronization Rule depend on. When selected, the Scope and Relationship pages are automatically greyed out. Once a Synchronization Rule is made to depend on another rule, the only settings that are adjustable on that rule are the workflow parameters and the outbound attribute flows. Conceptually, this falls cleanly from the fact that a dependent Synchronization Rule is being applied "on top" of another rule.

I wish I could paste some screenshots of what this looks like, but the FIM UI has changed markedly since the RC 0 release and I dont want to ruin the surprise just yet :)

The canonical scenario in which Synchronization Rule Dependency's are used are around creating business processes to manage the provisioning/deprovisioning of capabilities that stem from attributes set on a Active Directory user account. In a typical provisioning scenario, one would construct a base "Active Directory User Synchronization Rule" which, as the name implies, would create a new AD User object, flow the necessary base DN, samAccountName and name information. On top of that, you could then model a dependent Synchronization Rule for granting an Exchange mailbox. This Synchronization Rule would be dependent on the Active Directory User Synchronization Rule, and as a consequence would only have a single flow to the homeMDB attribute. Modelling the user account provisioning seperately from the mailbox provisioning, through the use of Synchronization Rule dependency, allows you to define independent business processes around the lifecycle management of the two through Management Policy Rules and workflow.

 As always, feel free to email me any questions you might have and I will do my best to get back to them.

Bobby

XPath Filter Dialect: Examples

The product team and I just wrapped up our week at the TechReady event in Seattle.  Bobby presented an excellent session on codeless provisioning, focusing on configuration and tips and tricks, and I presented a session on workflow and activity extensibility in ILM "2".  We also had the opportunity to solicit feedback about the product from attendees.  This event reminded me of just how new so many of the concepts in ILM “2” are, and how much more knowledge there is which can be shared.  My last post on the XPath Filter Dialect addressed one area where we frequently get questions, as our use of the xpath language is so pervasive throughout the product. 

While many of the common questions and areas of concern are fresh in my memory, I’ll proceed to share some guidance where I can. 

Let’s start with some examples that demonstrate the use of the XPath Filter Dialect addressing common queries (for reporting and other scenarios).  I’d recommend first reading the previous post on the xpath fundamentals.

Note: The XPath Filter Dialect is case sensitive.  Keep this in mind when writing your xpath filters.  For example, /Person[displayname = ‘value’]  is NOT the same as /Person[DisplayName = ‘value’].

Example 1:  A User’s Pending Approvals

You’ll need the following xpath if you want to build a report or page that lists all the approvals that are pending a response from a specific user.

Let’s assume the user, for which you want to see the pending approvals, has an Account Name of ‘mmeyers’ and an ObjectID of  ‘11111111-1111-1111-1111-111111111111’.

This first filter demonstrates how to identify the pending approvals based on the user’s ObjectID:

/Approval[ApprovalStatus = ‘Pending’ and Approver = ‘11111111-1111-1111-1111-111111111111’]

This second filter demonstrates how to identify the pending approvals based on the user’s Account Name:

/Approval[ApprovalStatus = ‘Pending’ and Approver = /Person[AccountName = ‘mmeyers’]]

Notice that in the second example we make use of a location path expression, /Person[AccountName = ‘mmeyers’], inside the predicate in order to identify approvals where the Approver is a user with the specified Account Name.

Note that the ApprovalStatus represents the status of an approval and can have one of the following values:

·          Pending

·          Approved

·          Rejected

·          Expired

A status of ‘Pending’ means the approval is currently awaiting a response from one of the users listed in the Approvers attribute of the approval.

A status of ‘Approved’ means the Request associated with the approval has been approved by the required number of approvers.  After an approval has been created it will only be marked as ‘Approved’ if the minimum number of responses, as specified by the ApprovalThreshold  attribute of the approval, is met.

A status of ‘Rejected’ means a user designated as an approver for the approval have rejected the approval.  At any point in time if a valid approver rejects an approval, that approval is immediately rejected and the workflow and associated Request is terminated.

A status of ‘Expired’ means the approval has reached the time indicated by the ApprovalDuration attribute on the Approval object as no response to the approval has been submitted.

Example 2:  All Security Groups expiring within the next 7 days.

/Group[Type= ‘Security’ and ExpirationTime <= op:add-dayTimeDuration-to-dateTime(fn:current-dateTime(), xs:dayTimeDuration(\"P7D\"))]

Example 3:  All Orphaned Security Groups

An ‘orphaned’ security group here refers to a group with no owner.  The following is the xpath to identify such groups:

/Group[Type = ‘Security’ and Owner != /Person]

Example 4:  People who are members of both the "Interns" group and the "Full Time Employees" group:

While my example here may not be a very compelling one, the goal is to demonstrate how we can identify users that are in sets or groups producing conflicting roles or permissions.

/Person[ObjectID = /Group[DisplayName = ‘Interns’]/ComputedMember  and  ObjectID = /Group[DisplayName = ‘Full Time Employees’]/ComputedMember ]

Note that I used the DisplayName attribute to identify the groups of interest, but the better practice would be to use a unique identifier to identify the groups, such as their ObjectID attribute.

Example 5:  People who were EVER members of both the "Interns" group and the "Full Time Employees" group at the same time:

The previous example identified people who are currently members of two conflicting groups.  The following example identifies people who were ever members of these conflicting groups at the same time.   This example makes use of the historical querying feature of ILM to scope the query to a time in the past.

allTime(/Person[ObjectID = /Group[DisplayName = ‘Interns’]/ComputedMember and ObjectID = /Group[DisplayName = ‘Full Time Employees’]/ComputedMember ])

Example 6:  All permissions that Kim Abercombie had in the month of January, 2009.

Again, here we see the use of historical query to check for a condition that was met at some time in the past.  This time we are checking for permissions that existed for a user between a specified time period.

betweenTime(/ManagementPolicyRule[GrantRight = 'True' and PrincipalSet = /Set[ComputedMember = /Person[ DisplayName = ‘Kim Ambercrombie’]]] , ‘2009-01-01T00:00’, ‘2009-01-31T00:00’)

Notice that the filter above is looking for any Management Policy Rule, of the type that grants permissions, which granted permissions to a set that contained Kim Ambercrombie in its membership

Example 7:  Changes to security groups in the last 10 days

/Request[Target = /Group[Type = 'Security'] and Operation = 'Put' and CreatedTime >= op:subtract-dayTimeDuration-from-dateTime(fn:current-dateTime(), xs:dayTimeDuration('P10D'))]

The above filter is returning all Requests that were created, within the last 10 days, to modify a security group.  If you want to find only the Requests that were actually ‘completed’, or the ones that were ‘rejected’ or still pending, simply add an additional condition based on the status of the request.

Example 8:  All full time employees that were ever contractors (ie. transitioned from one job to another).

/Person[EmployeeType=’Full Time Employee’ and ObjectId = allTime(/Person[EmployeeType=’Contractor’])]

Each of these examples is related to a question I've recieved in the past.  As more common scenarios become apparent, I will post examples addressing them. 

 - Nima

XPath Filter Dialect: Fundamentals

ILM “2” provides a Web Service Enumeration (WS-Enumeration) end point by which client applications can run queries and retrieve the results. Please refer to Joe Schulman’s excellent extensibility blog for more details on using the WS-Enumeration end point in ILM “2”.

This blog focuses on the XPath Filter Dialect, which you can use to create the queries to submit through the ILM Web Services.  XPath Filter Dialect is a subset of the XML Path Language (XPath) 2.0, with some additional functions.

Client applications can send WS-Enumeration Enumerate messages to the ILM Service to identify a of resources and attributes.  The subsets resources to return are identified by expressions in the XPath Filter Dialect (from here on usually referred to simply as an ‘xpath filter’ in this blog). 

If you are developing your own custom client for ILM, or submitting queries to ILM through a custom activity, you will need to use the xpath filter as your query language.  In the ILM portal, xpath is the language in which you express the queries for Search Scopes that can be created.  Xpath is also used to express the membership conditions of calculated groups and sets.

The following is an example of an xpath filter that identifies all people whose Job Title is ‘Engineer’: /Person[JobTitle = ‘Engineer’]

Rather than continue listing examples of xpath filters you may find useful, I’m going to use this blog post to describe the fundamentals of the ILM Xpath Filter Dialect so that you can understand the expression language and construct any filter you need.  I will follow up this blog with another post of some sample filters for commonly requested queries and reports.

Much of the content I include here is probably covered by the ILM “2” SDK in much more detail, but hopefully you’ll also find some of the guidance here conveniently available and useful J.  I will build on this topic with examples as people request them, so feel free to make suggestions!

Let’s start by looking at the data types the ILM xpath filters support.

Data Types

The ILM XPath Filter Dialect supports the four data types defined for XPath 1.0, plus the dateTime type that is defined in the XML Schema specification. These types are defined in the following table.

Now that we know what types of data we can filter on in our expressions, let’s take a look at what types of expressions we can actually define.

Types of Expressions

ILM “2” supports the following types of xpath expressions:

1. Location path expressions
2. Boolean expressions
3. Equality expressions
4. Relational expressions
5. Function calls

1.    Location Path Expressions

A location path expression identifies a node-set (collection of resources). A location path expression consists of one or more location steps. Location step expressions are delimited by a forward slash (/).  A location path expression must refer to an object type in ILM, or an attribute of type reference which refers to a resource.  Location path expressions have the following form:   /step/step/… | step/step/…

A forward slash at the beginning of an expression indicates an absolute location path expression as distinct from a relative location path expression. A relative location path expression identifies a node-set relative to the context node-set. The context node-set is the set of nodes that have already been identified.

Example: /Group/ComputedMember is a location path expression that consists of two location steps: Group and ComputedMember.  The result of this filter is all resources that are the ComputedMember of any Group.

Example: /Person[AccountName = ‘nima’]/Manager returns the resource referenced by the Manager attribute of the Person with an Account Name of ‘nima’.

Example: /Person[AccountName = ‘nima’]/DisplayName is not a valid xpath filter because DisplayName is not a reference type attribute.

            Union of location path expressions

The union of one or more location path expressions can be obtained by linking the location path expressions with the union operator, which is denoted by the vertical bar character, |.

Example:  /Person | /Group returns all people and groups. 

Predicates

Predicates are expressions that appear enclosed in brackets at the end of location steps. In the XPath Filter Dialect, predicate expressions must be Boolean expressions, equality expressions, function expressions or relational expressions.

Predicates filter the current node-set to produce a subset. A predicate is evaluated for each node in the current node-set. If the result of the predicate is true for a node, that node is included in the subset yielded by the predicate; otherwise, it is excluded.

Example:  /Person/Manager[JobTitle = ‘VP’] returns all people whose Manager’s Job Tile is ‘VP’.  The location step here are Person and Manager[JobTitle = ‘VP’] .  The second location step consists of a node name, Manager, and a predicate, JobTitle = ‘VP’.  

You can even have location path expressions nested inside predicates.

For example, the filter /Person[Manager = /Person[JobTitle = ‘VP’]] returns all people whose Manager is a person with a Job Title of ‘VP’.  Note that this returns us the same result as a previous example: /Person/Manager[JobTitle = ‘VP’].

2.    Equality Expressions

Equality expressions test the equality of terms. They have the following form: left_hand_term operator right_hand_term

The valid equality operators are as follows:

The left-hand term of an equality expression must be the name of an attribute in the ILM schema.

The right-hand term of an equality expression can be one of the following:

·         A function call.

·         A Boolean value.

·         A dateTime value.

·         A number.

·         A string.

·         A reference value.

If the left-hand term is a reference type attribute, the right-hand term can be a location path expression (ie. a filter representing a sub-condition).

Multi-Valued Equality Expressions

When the = operator is used in an equality expression where the left-hand term is a multi-valued attribute and the right-hand term is a literal value, the expression evaluates to true if the value of the right-hand term is any of the values contained in the left_hand_term.

Example:  /Group[ComputedMember = ’11111111-1111-1111-1111-111111111111’] returns all groups whose ComputedMember attribute contains the resource with the ObjectID ’11111111-1111-1111-1111-111111111111’. 

When the = operator is used in an equality expression where the left-hand term is a reference attribute (multivalued or single valued) and the right-hand term is a location path expression, the expression evaluates to true if the value of the attribute on the left-hand term is any of the values contained in the node-set returned by the right_hand_term. 

Example:  /Group[Owner = /Person[EmployeeType = ‘Contractor’] returns all groups whose Owner is a Contractor. In other words, this filter returns all groups where any of the values of their Owner attribute is among the set of people whose Employee Type is ‘Contractor’.

3.    Relational Expressions

Relational expressions compare the values of two terms. They have the following form: left_hand_term operator right_hand_term

Valid relational operators are: <=, <, >=, >, which are pretty self explanitory.

4.    Boolean Expressions

Boolean expressions evaluate the validity of two expressions in a predicate using ‘or’, and ‘and’.

When ‘or’ is used, the predicate evaluates to true if either expression is true.

Example: /Person[JobTitle = ‘VP’ or ‘Senior VP’] returns people whose Job Title is ‘VP’ or ‘Senior VP’.

When ‘and’ is used, the predicate evaluates to true only if both expression are true.

Example: /Person[JobTitle = ‘VP’ and Department = ‘Sales’] returns people who are VPs and are in the Sales department.

5.    Function Calls

The ILM XPath Filter Dialect provides the following functions that can be used in location path expressions:

Hopefully this blog helps you understand the structure of the XPath Filter Dialect expression language.  Trust me when I say this knowledge will come in extremely handy if you will be performing any of the following:

·          Creating search scopes for the portal.   Search scopes are pre-canned searches you can use in the portal.

·          Creating advanced calculated sets and groups that cannot be created using the Filter Builder control in the ILM portal.  One example of such a set is the set of all resources that contain an Expected Rule Entry for a particular Synchronization Rule in their Expected Rules List.

·          Creating custom activities that will query the ILM Service database.

·          Creating a WS client that will submit queries to the ILM Web Service.  Such clients can be used for purposes such as reporting.

Stay tuned as I will be following up with some xpath ‘tips and tricks’ and sample filters for commonly requested queries.

Cheers,

Nima Ganjeh

ILM "2" also comes as a hybrid...

For those of you who are MIIS / ILM 2007 pros, when seeing the Codeless Provisioning functionality one of the first questions that comes to mind is "can I use my existing rules extension in ILM "2"?".

Of course.

At a basic level, with ILM "2" RC, you can take an existing ILM 2007 deployment and migrate it's synchronization engine component straight into ILM "2" RC. You can do this by copying the IdentityIntegrationServer DB to a ILM "2" server, and upon installing ILM "2" point to this database instance during the setup of the synchronization component. The installer will then migrate that data forward such that all existing MA and MV configurations are ready to use right away, including rules extensions.

But if you want to go beyond this, its important to note how Codeless Provisioning works side-by-side with existing ILM synchronizaiton concepts. That is, while Codeless Provisioning bubbles up a business process driven approach to synchronization it is inherently underpinned by the same basic mechanics which power the ILM synchronization engine. As such, the adding of this functionality should not in any way change the behaviour of how MA's work, how rules extensions are called or how traditional metaverse provisioning is done.

This side-by-side coexistence is collectively referred to as a hybrid deployment.

Metaverse Provisioning 

In fact, it is supported to run Codeless Provisioning based provisioning logic side by side with traditional metaverse extensions. Codeless Provisioning is driven through the processing of Expected Rule Entry (ERE) objects, these determine which MV objects are provisioned a connector and how flows are applied on top. For a MV object being sync'ed, this processing is done prior to the calling of the Metaverse rules extension. Hence if for any reason the ERE's attached to a MV object do not achieve a desired outcome in a CS, you can use a Metaverse extension to provision additional connectors, apply initial flows and deprovision existing connectors just you would have done with ILM 2007.

Custom Functions = Rules Extensions 

Metaverse extensions are just one aspect of a hybrid scenario. A more common use case is for scripted flow. ILM "2" RC contains around 20 built in functions, which for the most part should satisfy most basic needs. However if this is not true, then you can always use an traditional Rules Extension to apply a transformation on a outbound flow. Using an MA, you can defined an advanced flow like before. This flow will be applied after any Sync Rule flows have been pushed onto an object, thus allowing you to append or overwrite attribute flow data that was provided by a Sync Rule.

Join / Projection Rules

On the inbound side, the traditional Join/Projection concepts live on as you remember them in ILM 2007. Just like the extension points, you can use traditional declared/advanced join projection rules along side Synchronization Rule concepts. In this case, if you have defined a Inbound Synchronization Rule on an MA that also has traditional join/projection rules defined the Synchronization Rule will be evaluated first. So if a disconnector exists within this MA such that it matches a Synchronization Rule's connected scoping filter, than this disconnector will be attempted to be joined/projected to the MV based on that Synchronization Rule definition. If the evaluation of that disconnector against the Synchronization Rule results in the CS object remaining a disconnector, then the existing declared join/projection rules will be executed against it.

How to make ILM "2" scream.

One of the big features of ILM "2" RC are the changes we have made to the portal and server to enable ILM to scale higher. In the portal and in the ILM service we've tuned, and jiggered with the way we interact with SQL and on the wire to make the ILM experience faster and snappier. Out of this, we've both gotten a better idea of what it takes to deploy ILM in such a way to maximize performance as well as key pieces of knowledge to help administrators keep their deployment running zipper quick. I will be writing a series of posts outlining much of this knowledge over the coming days.

Hardware

• If you are asking yourself which hardware to buy to deploy ILM, recognize that ILM is extensively CPU bound. Having a fast disk is essential, however the key gating factor is the CPU horsepower on your box. This is because of ILM's policy and rights calculations require SQL to execute multiple-statement operations when performing what are seemingly simple operations from the portal. Typically we use 4xQuad-Core boxes in our performance deployments.
• In an organization of 50,000 users, 20,000 groups, you should expect your base database size to be about 7 GB or so. The growth of this database will be dependent on the frequency of changes executed on the ILM system. Any rig set up for ILM should have enough RAM to load this database entirely into memory.

SQL

SQL is where performance starts and ends in the ILM Service. It is critical to having ILM perform at enterprise scales that SQL is setup such that it can best serve the ILM application. You may wonder why we took the hard dependency on SQL Server 2008 in RC, and the answer is more than it's snappy new logo. SQL Server 2008 introduces a new feature called Filtered Indices which specifically aims to limit index pollution as well improve queries across sparse columns by selectively including values within specified indices. ILM weakly-typed, single-table based storage mechanism begs for the usage of this new capability and we did it. If you only have 1000 objects in your ILM store, well then this isnt much of a deal, but when you start scaling into the 50k+ arena, filtered indices come into their own.

Beyond this, deploying ILM requires a steady eye towards maintaining and monitoring ILM performance. Here are couple of tips off hand which should help you keep ILM screaming:

• Turn off Automatic Statistics Updating. SQL uses a sampling technique by default to generate the statistics it uses for creating query plans. We've found that sometimes SQL grabs a bad sample and as such you will see queries all of a sudden fall off of a cliff. To prevent this turn off the automatic statistics update and manually run a fullscan statistics update on the ObjectsInternal table. Frequency of executing this will depend on the frequency of updates to the ILM database.
• Pre-grow your ILM DB, TempDB and Transaction logs. Ensure that you max out these files from the get-go so you do not take the cost of incrementally increasing the files during normal operation.
• Ensure you have seperated out the ILM DB, TempDB, and Transaction logs onto seperate drives.
• Ensure that SQL has been set to have a fixed upper limit on the memory it can consume during operation. If left uncheck you will see SQL gobble away memory until it starts to adversely affecting the performance of both the OS and as a result ILM.

ILM Service:

The ILM Service itself is actually quite lightweight and  very much dependent on the performance of SQL. To help streamline the service further you can try:

• Ensure tracing is set to error level or is off completely. Running at an excessive tracing level is guaranteed to bring ILM to it's knees.
• If you find certain queries are taking so long that the underlying SQL connections are timing out, you can use the dataReadTimeout and dataWriteTimeout attributes within the resource management service node in the resource management service configuration file to set the number of seconds for the underlying SQL timeouts.

That's all for now. Look for some further posts talking about other ways to monitor and manage ILM performance.

 In the meantime, we are currently at Tech Ed Europe in Barcelona. You can come find me, or Nima,  at the ILM booth located in the main exhibition area for the next couple of days. I will be running an Instructor Led Lab on ILM on Thursday at 1pm.

If you get a chance, I definitely recommend attending one of the many ILM sessions being done over the next couple of days:

Announcing Identity Lifecycle Manager “2” Release Candidate

We're back! Today we have a guest blogger to announce the Release Candidate for ILM "2":

I’m Lori Craw, group product manager covering identity and security at Microsoft. Today I am pleased to announce the exciting news that the Identity Lifecycle Manager “2” Release Candidate (RC) is now available.

ILM “2” builds on our Identity Lifecycle Manager 2007 investments, and includes solutions that will help IT more efficiently and effectively automate identity policies for users and their associated credentials and entitlements. An important set of features in this release focus on self-service for end users, enabling them to manage some of their own information like their passwords and groups, using self-service tools built into Office. The IT pro tools, combined with end-user self-service in Office and developer experiences using .NET and Visual Studio add up to a very powerful combination.

We’ve received great feedback on these features from our beta and TAP customers so far. We also have seen some great momentum with core partners, including Quest, Omada, and Unisys.

So what’s new in the RC, you ask? Some of the top new additions to the RC include:

- Support for scale out of the ILM "2" middle tier database and portal on separate servers, and support for multiple portal servers
- Added support for managing groups across multiple Active Directory forests
- Improvements in the request and notification emails, including customizable notification emails and request details baked into the out-of-box request emails
- Support for third-party certificate authorities (CAs)
- Performance and stability improvements
- Localization support for German and Japanese

I’d like to invite you to learn more about ILM “2” and to try out the RC for yourself by visiting www.microsoft.com/ilm2.

Regards,
Lori Craw
Group Product Manager
Microsoft Identity and Security

Custom expressions in advanced attribute flows

If you are at the point where you're comfortable with using the functions and various options in attribute flow creation, I'd like to introduce you to a really cool feature to add some efficiency and extra capabilities to your attribute flow creation:  Custom Expressions.

The synchronization rule designer provides you the ability to type in any attribute flow you can define using the flow definition user interface.  This is supported through the CustomExpression option you'll find under the advanced options in the value selection list when selecting a value for Source in an attribute flow.  See the screenshot below for clarification.

 So, what syntax exactly do you use to define the flows in this custom expression box?  Create an attribute flow in the normal manner (without using CustomExpression) and take a look at the attribute flow summary view when you save the flow, as shown below. 

 This summary view represents the attribute flow in a short form syntax as follows:

·         Constant strings are represented in quotes (“ “)

o   Eg. “Nima”

·         Constant numbers are represented simply as numbers.

o   Eg. 456

·         Attributes are represented as the attribute name.

o   Eg. FirstName

·         Synchronization rule parameters are represented as the parameter name prefixed with ‘$’.

o   Eg. $InitialPassword

·         Functions are represented as the name of the function and the associated parameters are given values that follow the syntax outlined here.

o   Eg. Left(“TestString” , 5)

·         The concatenate operator is represented as ‘+’.

o   Eg. FirstName + LastName

The syntax above is the exact same syntax you would use with the CustomExpression option.  The example below defines an attribute flow for generating a random password.

 Hopefully this makes defining flows a bit quicker for some of you.

The use of the CustomExpression option extends beyond just efficiency for power users though.  The CustomExpression option also allows you to define attribute flows containing nested functions, as you saw in the example above for generating random passwords.  In addition, you can also use CustomExpression with another very cool function available in the synchronization rule designer:  The IIF function.

 IIF function (Immediate IF) returns one value if a specified condition evaluates to true, and another value if the condition evaluates to false. 

The function's signiture is as follows:  IIF(condition, valueIfTrue, valueIfFalse).  -  Note that in beta 3 the function is mispelled as IFF :)

We use this type of functionality in programming all the time, so how does it provide value in the context of an attribute flow?  Well, think of any example where you'd want to flow a particular value to an attribute if some condition was true, and another if it was false.  One such example is defining the flow for an employee's email address.  Let's assume we want to prefix the email alias of a vendor/contractor with a "v-", and for full time employees we leave the alias as is. 

In other words, if the employee is a vendor/contractor, his email becomes his mailnickname "@microsoft.com" prefixed with "v-".  So, if nimag was a contractor, his email would become "v-nimag@microsoft.com"

In the synchronization rule designer, we can specify this attribute flow as follows:

For the condition, we used the CustomExpression option to allow us to specify an expression to evaluate.  Here we used the Eq() function which takes two arguements as inputs and compares them for equality, returning true if the attribute has the given value and false if not.  The condition is not very clear from the screenshot above, and its complete form is Eq(managed:EmployeeType, "Contractor").  The condition is checking to see if the managed:EmployeeType attribute has the value "Contractor".

Our return value if the condition ( the result of the Eq() function ) evaluates to true is the employee's email address prefixed with "v-".  The return value if the condition evaluates to false is the employee's email address as is (no prefix).

 The following are functions available for use as expressions in the IIF function:

Eq - This is the function used in the example above.  This function compares two arguements for equality.

NotEquals - This function compares two arguements for inequality, returning true if they are not equal, and false otherwise.  Example:  NotEquals(managed:EmployeeType, "Contractor")

LessThan - This function compares two numbers, returning true if the first is less than the second and false otherwise.  Example:  LessThan(Salary, 100000)

GreaterThan - This function compares two numbers, returning true if the first is greater than second and false otherwise.  Example:  GreaterThan(Salary, 100000)

LessThanOrEquals - This function compares two numbers, returning true if the first is less than or equal to the second and false otherwise.  Example:  LessThanOrEquals(Salary, 100000)

GreaterThanOrEquals - This function compares two numbers, returning true if the first is greater than or equals to the second and false otherwise.  Example:  GreaterThanOrEquals(Salary, 100000)

 Hopefully this helps bring to light some of the powerful capabilities you now have with the new provisioning features of ILM "2" !

- Nima

One DRE and a large shake to go...

For those of you who have used ILM "2" Beta 3, you have probably used in some form the new codeless provisioning functionality included within it. There is a ton of functionality encapsulated within this one area, and one of the less-talked about and centrally important pieces of this functionality is something we call the Detected Rules Entry (DRE). Do not confuse this with the Expected Rule Entry (ERE) object as they are two ends of two different sticks. The DRE very simply is an object that is created by the ILM "2" synchronization engine and associated to an ILM managed object when the synchronization engine detects that the flows as defined within a specific Synchronization Rule have been confirmed to exist within the connected system. More simply, the DRE is designed to provide the truth with regards to an object's state in a connected system, with the lingua franca in this case being communicated via definitions of logic which are Synchronization Rules. If the ERE can be thought of what we want the desired state of an object to be in a connected system, the DRE is the actual state of the object.

 How are DRE's created?

 You may have noticed within the Synchronization Rule designer a check box on the attribute flow page which says "Use as Existence Test?". When checked, the conjunction of all flows marked as being Existence Tests are evaluated by the synchronization engine against all connectors associated with any ILM object. This evaluation is done during synchronization of a management agent and obviously done on connector objects which are being processed as part of that synchronization run. If a connector space object is detected as having met the conditions of the Synchronization Rule, the synchronization engine creates a DRE object in the Metaverse, and places a forward link from the ILM Metaverse object to which the aformentioned connector object is joined to. From an ILM Metaverse object perspective, it has an attribute called "Detected Rules List", which is a multi-valued reference attribute to all DRE objects associated with it.

Ok, so why should I care?

Aha. This is the important part. DRE's allow you to create and launch business processes after a particular state is confirmed to exist within a connected system. (Think of needing to create a home directory after an AD user account is created)  DREs are only ever created based off of changes that are confirmed within the connected system (i.e. brought in through in an import), this allows you to then launch actions after having a particular state pushed to a connected system. After creation, DRE's are pushed via the ILM MA to the ILM Resource Management service. They are then subject to MPR and Process evaluation just like every other change coming to the web service.

So if we take an example of an Active Directory User Account synchronization rule. You may have anywhere from 5-20 flows for an AD User account synchronization rule. However, ask yourself this, what's the limited set of flows that you need in order to confirm  that a particular ILM object is associated with a confirmed AD user account? Probably 2, one for detecting the state of the userAccountControl attribute being set to 512  and the other matching the samAccountName on the user account with the managed:AccountName attribute on the Person object. By setting these 2 flows as existence flows within the Synchronization Rule designer, you can then trigger the creation of DRE's anytime the Synchronization Engine confirms those two flows on a connector object.

Some scenarios where this may be useful:

 - Triggering the granting of other out-of-band provisioning tasks that require an Active Directory user account to be present prior to launch.

- Compliance detection. DRE's are triggered on changes brought in from other systems. You can use DRE's to detect if somebody has an account in a system which was not granted via ILM, and then use MPR and Process to launch a workflow notifying their manager or an administrator of the existence of such an account.

Caveats in Beta 3:

- Existence flows cannot be defined for function flows and reference attribute flows

- Currently no mechanism to trigger workflow decisions based on the parent object of a DRE

Data transformation and attribute flow without writing code: functions to the rescue!

If you read the post on setting up the synchronization rule for the flowing of Computers to AD, you'll notice we make use of a concatenation option to concatenate multiple values and flow them to a destination attribute.  Concatenation is an example of a data transformation function that allows you to operate on and transform data you wish to use in the context of an attribute flow or workflow.  In ILM "2" you have over a dozen such functions available for you to use.  These functions are a critical aspect of "codeless" provisioning in ILM "2". 

In future posts I will dive into detail on specific functions and how you can use them to support your scenarios.  For now I want to introduce you all to this piece of functionality you may not be aware of.   To begin using these functions, either a)create an action workflow with the function activity or b)create an attribute flow for a synchronization rule.

The inclusion of functions in attribute flows means you can construct flows that involve more than just a simple attribute to attribute flow without writing any code.  For those of you that are familiar with writing scripted attribute flow in ILM 2007, you'll immediately appreciate some of these functions as they remove the need for writing custom code for many of the attribute flows you require.

It's easy to identify a case where you require the use of a function.  Perhaps you wish to generate a random password to flow for a new user account that is provisioned.  The screenshot below, from the synchronization rule designer, demonstrates the use of the concatenation and random number functions to create a password that consists of the string "Password" concatenated with a random number.  Of course we could define a more complex password, but this illustrates a simple example of how we can use functions.

Stay tuned for a detailed overview of specific functions.

 - Nima

The Mysteries of the ILM Management Agent Uncovered:

One of the many changes we made across the ILM to support the new declarative synchronization and provisioning concepts (aka 'codeless provisioning') was with the ILM Management Agent (hence forth referred to as ILM MA)  configuration experience. 

While at first glance the ILM MA may walk like, look like and act like any one of the many management agents that we have all come to know and love from the MIIS/ILM days, this resemblance is, at best, superficial. In fact the ILM MA is not your typical Management Agent, and has a unique design experience tailored to fit within the broader conceptual relationship between the ILM Application Store and the ILM Metaverse. While the two stores co-exist as independent stores in ILM "2", the relationship between the two should not be thought of  in the same vein as the typical relationship between a connected system and the Metaverse. Instead, it is envisioned that the ILM Application Store is in fact conceptually equivalent to the ILM Metaverse. Taking this notion one step further you should view the Metaverse serving as a transient storage mechanism on the road to and from the ILM Application Store, the ultimate  location for any data being synchronized into ILM. Seeing this,  we can then very easily imagine data moving from Active Directory or another third party system into the ILM realm and immediately coming under the control of the policy and process framework which applies to all data within the ILM Application Store and is at the conceptual heart of ILM application. (In order for this statement to be valid is for all data to make it to the Application Store)

With this view in mind, when we looked at the ILM MA, we wanted to make the experience of setting up the Management Agent between the Metaverse and Application Store to both provide a simple and "replication" like experience as possible. Thus, when you configure the ILM MA, you do not set up join and projection rules, nor do you need to write provisioning code or use Synchronization Rules to move data between the ILM MA connector space and the ILM Metaverse. Instead, you simply map object types from the Metaverse to object types in the Application Store, and then setup the attribute flow relationships between the two. After this point in time the ILM MA will automatically make sure that any new objects which are created in the ILM Application Store of the type specified in the mapping  are replicated to the ILM Metaverse as an instance of the second type specified in the aforementioned mapping (and vice versa).  The ILM MA thus allows customers the flexibility to connect newly defined ILM Application Store types to existing schema elements that they have created with their existing ILM Metaverse deployments.

Some of you may wonder if you are able to use scripted attribute flow and other extension mechanisms within  the ILM MA. The answer to this is no. The reason for this is that transforming data as it flows between the ILM Metaverse and the Application Store would violate the conceptual tenant that the two are logically equivalent. Instead, any desired data transformation should be done either upon inbound flow from a connected system into the Metaverse (via a Synchronization Rule or scripted flow) or in the ILM Application Store (as part of a workflow or through a web service call).

If we look at Nima's example of setting up the flowing of Computers, his configuration of the ILM MA demonstrates the new mapping functionality. You will notice in the ILM MA that the projection and join screens have been replaced by a Object Type Mapping screen. It is in this screen in which you map a Application Store type to its associated Metaverse type. (a known issue in Beta 3 is that your Metaverse type must be pre-fixed with "managed:" to be visible in the object type selection).

On the next page, you will then need to configure the attribute flows between the two types. ILM does not attempt to automatically map the attributes themselves and leaves it up to the user to determine how the attributes should be flowed. (Note: only direct flows are supported). The ILM MA also automatically adds a set of built in flows needed to support the replication functionality of the ILM MA, do not delete these!