Fluent Bytes

"The only source of knowledge is experience" - Albert Einstein-

Switching browser in CodedUI or Selenium tests based on MTM configuration part 2

More then a Year ago I wrote a blog post on how you could switch browsers based on the configuration settings in MTM. You can read more about that in this post here:http://bit.ly/1AKeFAS

So this solution unfortunately works partially and I have talked with the TFS team about this for a while but this is something that might take a while to get fixed. This week someone asked me again about this solution and I worked on a work around to get things running the way we need.

So first, what is the problem?

When you run one test case all will run fine. the Test Context will contain the right information and you can use the configuration name that is passed in the __Tfs_TestConfigurationName__ property from the TestContext.Properties collection.

But when you do a multi select of some test cases you want to run or you run a full test suite, you will run into the issue that the information being passed in the TestContext Properties collection becomes stale after the first test case is done. So if you look at the screenshot below, you can see that I picked 3 test cases to run:

image

When you look at the results (this test just dumps out the properties in the collection to see what is in there) you can see that for each run the configuration has the same value Confused smile:

imageimageimage

After reporting this back to the team, this was confirmed as a bug and was not fixed in the recent versions and updates.

How to solve the issue?

So how to solve this problem, so we can really switch the browsers based on the configuration in MTM?

the post here really helped me on my way to get this issue fixed: http://blogs.msdn.com/b/aseemb/archive/2012/08/07/code-snippets-on-test-management-apis.aspx (code snippet #7)

The solution is that we need to query the TFS server on the current test run and based on the current __Tfs_TestRunId__ that is the same for all iterations. Then we can ask for the Test Results for the current test that is running on the current machine. In the test results we can then get the Configuration information of the current running test and that will reflect the real configuration of the current iteration of the test we are running.

to make this work I needed to write up a bit more code then I anticipated, because the Test Result data is reported back to TFS once every 30 seconds and when you start a new run, you might get the data of the previous run, since TFS might lag behind a bit. So I needed to come up with a waiting pattern and only get the data when the test identifier changes, so I know we are in a new fresh run. The code I wrote makes one big assumption and that is that the tests are run in the same process, since I am using a static variable to contain the last test id. This is not pretty code, but it gets the job done.

Here is the code on querying the server and get the right configuration data:

After you have this code, you can now use this function to switch the browser for you in your test by passing it in the Testcontext.

For example in codedUI you can switch the browser now as follows:

BrowserWindow.CurrentBrowser = GetRealTestConfigName(TestContext);

If you now look at the results of the same test run you can see the code finds the right configuration. You can see in the screenshots below it will now really find the actual configuration we are running.

imageimageimage

One issue I found with this solution is that your tests will experience a little delay, because we need to wait for the test run to show up in the TFS server, before we can query the data. But if you look at this you might not really find it a problem since it normally is only a small portion of the time spend on a test. These tests are not meant to be a fast unit test, so I guess we can live with it for now.

I sure hope this will eventually get fixed in the test infra structure. If that happens I will post back with that solution.

Hope this helps,

Marcel

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

2 Comments

  1. Marcel,
    There is a variable missing: lastTestPointId
    As you reply in the other blog. I tried to declare the following variable that you mentioned, but it says that “testCaseResult” needs to be declared before.
    So, my question is, its this variable that i should declare? And what part of the code?

    To try to solve that I declared the lastTestPointId variable as an int and after this line int id = results.First().TestPointId;
    I included:
    lastTestPointId = testResults.Last().Id.TestRunId;

    The error doesn’t show up any more. But I didn’t tried to execute to see if it works yet.

  2. Thank you , this is really helpful! I’m going to try this out on my new projects.

    Please share the news as soon as you hear this issue is being addressed by Microsoft.

    Keep up the great blog posts.

Leave a Reply

Your email address will not be published.

*

© 2017 Fluent Bytes

Theme by Anders NorenUp ↑