Fluent Bytes

"The only source of knowledge is experience" - Albert Einstein-

Switching browser in CodedUI or Selenium tests based on MTM configuration

In my previous post I have shown how you can integrate Microsoft Test Manager(MTM) with Selenium for cross browser testing of your web applications. One of the issues I did not touch what how can you switch to the correct browser based on the configuration selected to run in the Test Suite.

So in MTM  you can configure the configurations you want to test. For a typical web scenario the following test settings would be a good example:

image

Now when I run a Test Case from the MTM environment, the test case will be put to the ready state for each configuration. Next I can run those test cases and from the Test Tab and the associated test automation will be started.

Now in order to adjust my test to pick the correct browser, I want to know what configuration this test is running under. With Update 1 of visual studio 2012, the test controller and test agents, you can now get this information from the test context. Below you can see a list of properties that are available when you run a Test that is triggered from Microsoft Test Manager.  (this is the TestContext.Properties[] dictionary)

Variable Name Example Value
TestRunResultsDirectory C:\Users\Administrator\AppData\Local\VSEQT\QTAgent\22\TestClient\Results\TestClient
AgentId 1
AgentName TestClient
TestRunDirectory C:\Users\Administrator\AppData\Local\VSEQT\QTAgent\22\TestClient
BuildDirectory \\vsalm\ffdrops\New Build Definition 1\New Build Definition 1_20130222.7
DataCollectionEnvironmentContext Microsoft.VisualStudio.TestTools.Execution.DataCollectionEnvironmentContext
TestResultsDirectory C:\Users\Administrator\AppData\Local\VSEQT\QTAgent\22\TestClient\Results\b23878fe-9ce5-4444-83c1-563801f07762\TestClient
TestLogsDir C:\Users\Administrator\AppData\Local\VSEQT\QTAgent\22\TestClient\Results\TestClient
TestDeploymentDir C:\Users\Administrator\AppData\Local\VSEQT\QTAgent\22\TestClient\Deployment
ResultsDirectory C:\Users\Administrator\AppData\Local\VSEQT\QTAgent\22\TestClient\Results
ControllerName vsalm:6901
TestDir C:\Users\Administrator\AppData\Local\VSEQT\QTAgent\22\TestClient
AgentLoadDistributor Microsoft.VisualStudio.TestTools.Execution.AgentLoadDistributor
DeploymentDirectory C:\Users\Administrator\AppData\Local\VSEQT\QTAgent\22\TestClient\Deployment
TotalAgents 1
AgentWeighting 100
FullyQualifiedTestClassName UnitTestProject1.CodedUITest1
TestName CodedUITestMethod1
__Tfs_BuildUri__ vstfs:///Build/Build/18
__Tfs_IsInLabEnvironment__ True
__Tfs_TestRunId__ 22
__Tfs_TestCaseId__ 117
__Tfs_TeamProject__ MyTeamProjectName
__Tfs_BuildDirectory__ \\vsalm\ffdrops\New Build Definition 1\New Build Definition 1_20130222.7
__Tfs_LabEnvironment__ <?xml version=”1.0″ encoding=”utf-16″?><LabEnvironment Id=”5f37b167-ad24-4f7e-bb1e-2e65a3e71a1f” Name=”Windows 7 Client” Uri=”vstfs:///LabManagement/LabEnvironment/2″><LabSystems><LabSystem Name=”TestClient” ComputerName=”TestClient” Roles=”Desktop Client”><CustomProperties /></LabSystem></LabSystems></LabEnvironment>
__Tfs_TestConfigurationId__ 2
__Tfs_TestPlanId__ 4
__Tfs_TestConfigurationName__ Chrome
__Tfs_TestPointId__ 12
__Tfs_TfsServerCollectionUrl__ http://vsalm:8080/tfs/fabrikamfibercollection
__Tfs_BuildPlatform__ Any CPU
__Tfs_BuildNumber__ New Build Definition 1_20130222.7
__Tfs_BuildFlavor__ Debug
__Tfs_BuildConfigurationId__ 22

In this list you now see the __Tfs__xxxx properties that are available. Here you can now find properties on e.g. the Environment you are running in and the information from MTM which test plan and configuration this test is part of.

Now for running cross browser tests this is a huge benefit, since we can now leverage the configuration names of the tests that we run. In my particular case I can access the __Tfs_TestConfigurationName (highlighted in bold in the table) and this will return one of the three strings I have, “IE”, ‘Firefox” or “chrome”.

So what I do is set up the browser I want to run my test on in the [TestInitialize]. If you are using Selenium you can use the info to switch the driver, in CodedUI this is done by setting the static property on the BrowserWindow. the following snippet of code shows how this is done:

Selenium:

CodedUI:

So now you see that you can switch the browser based on the configuration you set in MTM.

Next is to set up a new Lab Management build, where you select the Build, Test Environment and the Test plan you want to run. Now you create a new Build definition for each configuration (you can only select one per build definition) and you can schedule them all in parallel since they will be queued to the test agent running in the environment.

This results in a set of test runs each morning that you can now check on any faults that night in any particular browser. You can see in the screenshot below taken from MTM, details of the run like: the build it ran on, if it was passed or failed and the time the run was made. Clicking on one of the rows, will take you to the results of that run, so you can further analyze what perhaps went wrong. Based on the Test settings you choose for the run, you will have more or less diagnostics to follow up in case of a failure.

image

Hope this helps you getting started!

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

6 Comments

  1. This is a good post.

    Nice hack for something that should work out of the box …

    I am puzzled how many flaws there between MTM and Visual Studio unit tests. I cannot believe they cannot talk to each other well.

    I really was expecting that if I define configuration and it has variables, I would be able to access them in the test.

    Seems like I can access only the config id and name, thus eventually making tests to be able to run parameterized with single parameter – the name of the config

  2. When I execute my test from MTM, context.Properties.Contains(“__Tfs_TestConfigurationName__”) always returns false. When I am in the Run Tests tab in MTM, the test case displays “Default Run on Firefox” under configuration. Is there an additional step I need to take for the “__Tfs_TestConfigurationName__” key to appear in the context.Properties[] dictionary?

    MTM About Dialog Info:
    VS/MTM: 2013; Version: 12.0.31101.0

  3. I’ve written a NuGet package based on the contents of this article that will use the values provided by MTM to connect to TFS and pull down the Configuration variables associated with the Test Run Configuration, so that you don’t have to base configuration values solely off the name of the Configuration (which is really not a good idea anyway). I agree with the rest of the commenters, stuff like this should really have been supported better by Microsoft. Hopefully somebody other than myself will get some use out of it. It’s available here: https://www.nuget.org/packages/TfsLabAccessor2013/ , source code available on github.com at https://github.com/alexmarshall132/tfslabaccessor2013

  4. Hi,

    Are these TFS data available when running the tests as part of vNext build as well? I am using TFS 2017 and TestContext[“__Tfs_TestCaseId__”] returns null. I made sure I associated the test case to the test method.
    I need to get the id of the running test case to be able to write some notes back to the test case result “Comment” section via code. I did that before using TFS 2013 (by queries) when we used to run the tests from MTM, which are now deprecated. I am trying to do the same thing using the new TFS API. I would appreciate any help on this.

    Thank you so much in advance.

Leave a Reply

Your email address will not be published.

*

© 2017 Fluent Bytes

Theme by Anders NorenUp ↑