Updated Reporting and Analytics Capabilities

Updated Reporting and Analytics Capabilities

  • Comments 4
  • Likes

Howdy folks,

In a blog post last November I introduced some new reporting capabilities. Since then we've been hard at work enhancing this set of features. Our goal with this work is to enable you to manage and protect your organization by answering some key questions, for example:

  • Are my users' accounts secure and which user accounts are at risk?
  • How much are our SaaS apps being used and who is using them?
  • What devices are my employees using to access SaaS apps?

Ruchi Chopra, the Program Manager who is driving Reporting and Analytics, will give you a tour of the functionality we've released so far.

I hope you'll find these new capabilities valuable. We're quite excited about the amazing level of visibility they provide you!

And as always, if you have any feedback or suggestions, we'd love to hear from you.

Best Regards,

Alex Simons (twitter: @Alex_A_Simons)

Director of Program Management

Active Directory Team

 -------------------------------------------------------

Hi everyone,

We've released some exciting reports over the last few months, and I'm particularly thrilled about some of our advanced reports that incorporate machine learning. Of course, we're continuing to invest in this area, and this is just a taste of things to come.

Our initial reports are in three categories that are important for organizations: ensure user account security, manage their use of SaaS apps, and monitor use of cloud-connected devices. You'll find most of them in the Reports tab.

Reports tab

 

Helping secure your users' accounts

Let's start with a story. Meet Audrey Oliver, a Project Manager at Contoso. Contoso has strong password requirements so that they can keep their enterprise resources secure. So Audrey has to create a password with at least 8 characters, uppercase and lowercase letters, a special character, and a digit. In other words, it's a strong and awfully hard to remember password. She doesn't want to keep track of too many passwords like this one.

Like most of us, Audrey uses a lot of websites- social, shopping, banking, etc. And to her delight (and her IT admin's dismay) she can just reuse her Contoso username and password everywhere.

No prizes for guessing what happens next. One of these sites gets attacked, and now her Contoso username and password are published on the internet. And guess what: as we speak, a hacker in a distant country is accessing Contoso's resources…and neither Audrey nor the IT department are aware of it.

The bad news- such incidents are becoming quite common these days. The good news- Windows Azure Active Directory enables you to view your organization's users' sign in activity, and it tells you if something looks anomalous.

What kinds of events does Windows Azure AD look for?

Our goal is to make you aware of anomalous activity and to enable you to be able to make a determination about whether an event is suspicious. We provide reports based on various criteria, which are described below.

The following reports are available as part of Windows Azure AD, and are available to all customer today:

  • Sign ins from unknown sources

    These are sign ins originating from a source that has been anonymized. It may indicate an attempt to sign in without being traced.

     

  • Sign ins after multiple failures

    These are successful sign ins that occur after several failed sign ins using the same account. This may indicate a brute force attack.

     

  • Sign ins from multiple geographies

    These are sign ins from geo-distant locations that occur within an interval that's less than the time taken to travel between the locations. This may indicate that multiple users are signing in from the same account.

     

We also have the following enhanced reports as part of Windows Azure AD Premium:

  • Sign ins from IP addresses with suspicious activity

    If we observe a high percentage of failed sign ins from the same IP address, it indicates that there may be a sustained intrusion attempt. Any subsequent successful sign ins from these IP addresses are flagged as anomalous.

     

  • Irregular sign in activity

    We employ machine learning to identify patterns in user sign ins, on the basis of criteria such as location of sign in, sign in activity of their peers, devices used, and time of sign in. Sign ins that fall outside a user's sign in patterns are included in this report.

     

  • Users with anomalous sign in activity

    We provide you with a consolidated view of all users in your organization that have had anomalous sign ins. This helps you get a holistic view of the anomalous activity in your organization.

     

Irregular sign in activity report

 

 

How do I know when to look out for issues?

We'll let you know! We've added support for email notification in Windows Azure AD Premium Preview. We've implemented an email notification (currently in preview) that will be sent to all the Global Admins when we detect 10 or more irregular sign ins in a span of 30 days. Here's what the email will look like:

Notification of irregular sign ins

What do I do if I see events in one of these reports?

Here's our recommendation:

  1. Contact the user to get more information about the activity.
  2. If you suspect that the activity is malicious, you can change the user's password, and tell the user their new password. Also, if the user is a global administrator or if you have Multi-Factor Authentication, you can also enforce Multi-Factor Authentication for added security. Look for these commands in the command bar at the bottom of the page. Just select the user in the report, then click on the command!

Insights into SaaS application usage

Windows Azure AD enables you to integrate cloud-based applications (we support over 1000) so that your users can single sign-on into them from the access panel (https://myapps.microsoft.com). What's more, we provide insights to the admin on how these applications are being used.

On the application Dashboard, you can see usage data for an application, aggregated across all your users. It shows you trends in usage (based on access attempts from the access panel).

Application dashboard

 

But that's not all. We also have a detailed report that tells you which users are most actively using an application, as part of Windows Azure AD Premium. This helps you gain insights into whether your organization is getting the right ROI for each application.

Application usage report

 

Devices employees use to access SaaS Apps

We also have a User Devices report that indicates the devices we've seen a user sign in from. This is also part of Windows Azure AD Premium. You'll find this under the Users tab. When you select a user, you'll see the Devices tab. In addition to information about the device, we include the most recent sign in activity observed from the device.

User Devices report

Retrieving report data

If you would like to use the data in any report in other tools or applications (such as Excel), you can also download it in CSV format. Just look for the Download button in the command bar at the bottom of the page.

Tell us what you think!

We'd love to hear from you. Your inputs help us ensure that we deliver a solution that works for you. If you have any suggestions, questions, or comments, please use this forum.

-Ruchi

Your comment has been posted.   Close
Thank you, your comment requires moderation so it may take a while to appear.   Close
Leave a Comment
  • I like the EMAIL NOTIFICATION OF ANOMALOUS SIGN INS, but I think sending the email to All Global Admins twice (as it goes to their O365 email address and their Alternate email address) is a bit too much.
    We should be able to choose or assign a Group that these get send to instead of All Global Admins.
    Regards
    Mark.

  • Thanks for your feedback, Mark! We're using the alt email address to ensure that the email is received, even in case the admins don't have Office 365. But you have a good point and we'll investigate other options.

    Mail to an assigned group instead of global admins is an ask several customers have made, and we're looking into how we can best address it.