With the Beta release of Global Service Monitor (GSM) we added one of the most requested features: ability to execute multistep web tests with authentication.
This is a big step forward compared with single URL tests we had in CTP, because it allows monitoring health of end-to-end user scenarios. Imagine your application sells some goods: say books or food (or groupons :-) ). Your users need go to the home page, login, possibly perform a search, place the item in the shopping cart, and check out.
Visual Studio Ultimate provides capability to record all those steps while they are being performed in a browser, and play them back. During the playback, various parameters about the test execution are being measured and compared for correctness with the criteria specified in the recorded test. For example, you can verify that each response is faster than certain time limit; you can check whether a certain string or a certain tag is present in the response, or if response is a redirect, you can verify the URL it redirects to. In essence, Visual Studio web test provides many knobs to fine-tune the criteria which indicate the success of your end-to-end scenario. Please see this walkthrough for more details about recording and configuring Visual Studio web tests.
Please keep in mind that GSM imposes certain restrictions on each Visual Studio web test to ensure that the test can co-exist well with other web tests which are running on the service. Specifically, we enforce the following limits in Beta release (these conditions are subject to change in the future):
Developers of the application will probably create web tests to verify performance and correctness of the end-user scenarios while they are preparing to roll out the application to production. These tests can be shared with the Operations Manager admin to use for monitoring of the application health. Operations Manager needs to have Global Service Monitor management packs installed and the service configured in order to use those tests in production.
Global Service Monitor can distribute the tests to the specified locations around the globe: currently we have 16 locations. The results of the test execution are analyzed for failure, and the pass/fail results of the tests are presented in the Summary Dashboard of the Global Service Monitor. Please note that you can configure more than one test, if your application supports multiple end-user scenarios. The results of all the configured tests will be rolled up into the health state of the application, which is displayed on the map (Summary) dashboard.
If something goes wrong in production and the tests start failing, Operations Manager will bring up an alert. Alerting behavior in GSM can be configured to alert on each web test individually, or to roll up all the tests and alert once for the whole application if at least one of the tests failed. You will get a reason why the test failed in the Alert Context. For example, if a string match validation rule fails, you may see a description similar to this:
Often, in order to fix the problem with the application, input from the developers is required.
With the TFS synchronization functionality available in OpsMgr 2012 SP1 Beta, we are completing the loop of DevOps communication: now the Operations Manager admin can assign the alert back to the developer for debugging and fixing the problem. Enabling running web tests recorded in Visual Studio and ability to assign OpsMgr alerts back to engineering makes DevOps integration and communication more effective and more informative. Both developers and operation admins can use the tools they have always been using and simultaneously get access to the full context of the issue. TFS synchronization provides two-way updates between OpsMgr alerts and operational issues in the TFS database, so that both parties could remain on the same page regarding the progress of the investigation. Such agile communication may help reducing MTTR.
We hope that ability to run web tests recorded in the Visual Studio from GSM locations will help companies to leverage existing dev assets (recorded tests) for the monitoring purposes. Multistep tests with authentication will enable in-depth monitoring of main end-user scenarios to ensure customer satisfaction, and ultimately will help running a successful business!
Please read more about this functionality in Manish's blog here: blogs.msdn.com/.../global-service-monitor-beta-relase.aspx
How about expanding the capacity for the number of beta subscriptions?
You asked - we listened. :-)
Please use this link to fill out the survey and start using GSM Beta.
Please keep the feedback coming!!!
Does the solution allow to run webtest from "agents" located in the companies internal network? e.g. one of the VS Agent which is a member of a Visual Studio Loadtest/Webtest RIG?
The new version of TFS is so awesome. We love using the preview. Can't wait to see all the cool new things in the final version.
Matt Watson, Founder of <a href="http://www.stackify.com">Stackify DevOps</a>
Matt Watson, Founder of Stackify
Ralf: current release only supports running VS web tests from external (GSM) locations. We are considering enabling internal locations in the future. Can you please explain your scenario?
Matt: thank you very much for your kind words!
Updated the number of allowed steps per test to 100.