Fluent Bytes

"The only source of knowledge is experience" - Albert Einstein-

Category: Uncategorized (page 2 of 14)

How to prevent CodedUI tests from breaking Angular Ajax calls

During one of the Pluralsight study group nights, we came across a problem when we test angular websites that make Ajax calls to switch templates in a Single page application. To repro the scenario we created a simple website that did nothing more then showing on the main page an button and hocking up that button with an event to switch the page with an second page, using an Ajax call.

The pages look like follows:
image

After clicking the button you will see

image

The code looks like this:

Index.html:

image

and Edit.html

image

and index.js:

image

Now the CodedUI test we ran looks like follows:

image

We launch the browser, Search for the button, then click the button. On the new page we search for the HtmlEdit box and read out it’s value.

Now this test works great as long as the Index.js file contains the following line to switch the page:

image

If we use an Ajax call by using the following code then the second page will not be shown in the browser. So it looks like the click is not executed.

image

What happens is that codedUI injects java scrip the moment it starts a browser test. This script is used to enable codedUI to wait for Ajax calls to complete and coordinate other timing related things.

Unfortunately this script seems to interfere with the standard way Angular is handling Ajax calls and therefore breaks your site.

Now the big question is how can we solve this?

Fortunately there is a simple solution. You can add an App.config file to your codedUI test project inside this config file you specify a configuration key called WebWaitForReadyLevel and set its value to 3. So your config file must contain the following xml:

image

Now when you run CodedUI will not inject the JavaScript in your page and this will then fix the problem of Ajax calls not working in your site.

Of course this has as side effect that CodedUI will not test the ready level of controls, so in order to ensure you can interact with controls you might need to add some control flow logic to the code to wait for controls to appear like e.g. the WaitforControlEnabled or WaitForControlExists that are available on all the test controls you use.

Hope this helps,

Cheers,

Marcel

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

Testing Angular sites with CodedUI (part 2)

In my previous post I showed primarily how you can select elements based on the ng-* attributes when using CodedUI. In this post I will complete the scenario on how we can test the sample application created by my colleague MVP Maurice de Beijer, who is an Angular expert.

The Scenario

The scenario we will walk through is the following:

Goto the site http://rawstack.azurewebsites.net/, Click on the Admin link, enter a movie in the filter box, Click on one row and then read out the audience score value and assert on the value.

Below you see the steps including the screenshots so you can follow along the steps how to implement this.

Goto the site and click the Admin link

image

Enter text in the filter box:

image

Click the row with the entered text we are searching for:

image

Now check if the value in the text box is 94

image

So what are the steps we need to code in CodedUI

.We need to get started with going to the Url and then click in the admin link. this is done with the following steps:

The next step is that we want to find the filter box, as described int he previous post. The challenge we have here is that this box is not available right away. What happens is that the site goes out with an Ajax request to get a list of data and when the data is returned shows the grid with the data and the text box where we can type the filter text.

To show progress the page has an progress indicator that shows an Ajax request is ongoing and this indicator is hidden the moment the Ajax call is done.

image

We can make use of this to wait for the call to complete.  We do this using the WaitForControlNotExisits call on the TestControl. This is done as follows:

Now we can search for the filter control and we can ensure we will find it.

Finding the control is done with the steps where we will use a JavaScript execute call to find the control using the document.querySelector and find the control by it’s ng-model attribute.

This is done as follows as seen in the previous post:

So now we have filtered down the list to the number of items we are looking for and next is to find the correct row we want to click

This part is a bit tricky, since only selecting the Span that contains the text item and clicking it, will not yield the right results. If you look at the table structure for the grid you see it has multiple div’s and you need to actually select the row div that contains the click handler to trigger the click that a user would normally trigger. If you look with the F12 tools on any of the browsers you will see the following structure:

image

So to find the correct row we can simply use the power of CodedUI searches. We only need to add two SearchProperties to find the correct div and then we can click it. We need to search for both the InnerText and the CssClass on the Div and use the PropertyExpressionOperator.Contains. The code for this is as follows:

After clicking the row we now will see the admin page for this row and we can finally search for the audienceScore input. This control has an Id, so the search for that is very simple and robust. Reading out the value and then asserting on that is nothing more then reading the Text property and asserting on that value. The code for that is as follows:

And that is all to make it work.

Implicit wait on Ajax by CodedUI

One final note is that CodedUI injects a small piece of JavaScript into you application on startup in order to be able to check if all UI threads and Ajax calls have completed. In this application this resulted in the Ajax busy indicator to keep spinning.  I removed the need of CodedUI to do the wait implicitly, by the explicit wait I do for the Ajax call to complete. So I added the following line of code just before I start the browser window:

This prevents the JavaScript to be injected and this solved the spinner being stuck in the application.

Conclusion

So you see that there are primary tree caveats in testing this angular application. The first one is waiting on the Ajax call to complete without relying on the default codedUI WaitForReadyLevel. We used the WaitforControlNotExists to wait for the operation to complete. The second one is finding controls based on the ng-* attributes in stead of unique identifiers, which we solved by using the JavaScript execute method on the BrowserWindow class. The last one was selecting the right Div on the Angular grid Control so the click is actually executed.

The solution I showed here works cross browser, so by just setting the BrowserWindow.CurrentBrowser = “crome” or “firefox” you can run this test also on chrome or Firefox with the same results.

Hope this helps,

Marcel

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

Testing Angular sites with CodedUI

Last week we hosted the first of a series of Pluralsight Study group meetings at our company Xpirit together with the DotNed user group. The study group focuses on the new training I just got published on pluralsight called “Testing Web Applications with CodedUI”.

In the first few chapters we covered last week we looked at how the CodedUI object model works and as one of the hands on assignments we thought it would be fun to test an Angular sample website created by my fellow MVP Maurice de Beijer that he uses in his training on Angular. (http://rawstack.azurewebsites.net/)

The site we wanted to test with CodedUI is located here and shows a list of movies as an infinite scroll list and it has admin pages where you can filter the list of movies and then click one of the rows to edit the movie details.

The scenario we wanted to test was as follows:

Goto the site, Click on the Admin link, enter a movie in the filter box, Click on one row and then alter the rating.

In my course I always urge people to find unique identifiers to find the controls you want to interact with, but as Maurice showed me with his Angular website, there is no use of the id attribute in any of the pages. This seems to be a common practice and therefore we need to find some other ways to uniquely identify the controls we want to interact with so we can find them on the screen.

With Angular it is very common you use attributes like: ng-model, ng-controller, ng-repeat, ng-submit, etc. So there is a way to uniquely identify these controls we just need another way to find them then using an id attribute.

Basically there are two options you have to achieve to find the controls with CodedUI. the first one is the one that most closely matches the CodedUI object model and the way CodedUI finds controls. We need to specify search properties and additional filter properties to find the control. The problem with this approach is that it is not working when you are using the cross browser plug-in and want to playback on chrome or Firefox. (what happens is that all filterProperties are ignored except taginstance, meaning you will most likely have a match that can contain multiple objects in stead of just one.

Since it is a common requirement to test cross multiple browsers this options is only useful for those who only test on the IE browser. Here is a code sample on finding the Filter control on the Admin page (http://rawstack.azurewebsites.net/MoviesAdmin#/moviesAdminList)

 

The second way of implementing this search, is by utilizing the ExecuteScript method on the BrowserWindow class. We construct a simple piece of JavaScript that runs in all browsers where we use CssSelectors to find the unique match on the ng-* attributes and then return the right type of Html control. You can see the code here below:

 

Great thing about the ExecuteScript method is that it returns a correctly typed object so you can interact with the control with it’s provided convenience properties that make interaction with the controls more easy.

With this way of finding the control, we also ensure it works cross browser, since  the ExecuteScript method is implemented for all supported browsers in CodedUI. You can apply any Css selector that is supported by the document.querySelector method. see here for more info on selectors: http://www.javascriptkit.com/dhtmltutors/css_selectors_api.shtml

In my next post I will show you the full steps on testing the full scenario.

Hope this helps,

Marcel

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

Upcoming speaking engagements

Then next two months I will be rather busy again with several talks I will deliver round the globe.

In May I will be at Techorama in Belgium where I will deliver to talks on Application Lifecycle Management.

I will do the following talks:

Using the Cloud-Based Load Testing Service and Application Insights to Find Scale and Performance Bottlenecks in Your Applications

Best practices for open source in the enterprise

End of May I will be at Techdays in The Hague. Schedule is not final yet, but you can find the sessions over here:

http://www.microsoft.com/netherlands/techdays/programma.aspx

In June You can find me in San Francisco at Visual Studio Live!

SFSPK07

Here I will deliver the following talks:

Workshop: Native Mobile App Development for iOS, Android and Windows

Best Practices for Using Open Source Software in the Enterprise

Adding Analytics to Your Mobile Apps on any Platform with Microsoft Application Insights and Hockeyapp

If you would like to join me in San Francisco, you can get a discount on the conference fee. Use this link to register and you will get a discount of $400. You can also use the following promo code if you don’t want to click the link: SFSPK07

Hope to see you soon at one of these conferences.

Cheers,

Marcel

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

Sessions VSLive Las Vegas

HI, thanks for all that attended my sessions at VSLive this week in Las Vegas. I hope you enjoyed the sessions and if you have questions left, please reach out to me on twitter (@marcelv) or ask them here below in the comments section.

WP_20150316_08_36_36_Pro

If you want to have a look at the slides ore want to use the demo’s your self, you can find them here below.

T23 Automated Cross Browser Testing of Your Web Applications with Visual Studio CodedUI

Slides

Demos

W05 Using Microsoft Application Insights to Implement a Build, Measure, Learn Loop

Slides

Demos: no demo’s to share, just new up a ASP.NET project and work it from there.

W30 Advanced Modern App Architecture Concepts

Slides

Mobile Workshop: Native Mobile App developement for iOS, Android and Windows using C#

For the one day mobile workshop, you can find those back on the VSLive download portal. Details on where the portal is and login details are in your VSLive conference book.

Cheers,

Marcel

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

Hosting CodedUI in your Wpf Application

Today I was asked an interesting and challenging question regarding CodedUI. We are working on some code that a colleague of mine wanted to test with Spec flow and CodedUI. The challenge was that in the spec flow test they created an instance of some WFP UI components that are part of the application and they wanted to use CodedUI to validate the behavior.  So this means the applciation under test is the test itself, since that is hosting the control.

So Why is this challenging, you might ask? Well there are a couple of things that will bite you when you try this and you need to get them solved first, hence my post to directly show you the solution 🙂

To simplify the scenario and show what you need to do, I created a simple test application that creates a Wpf Window. In this window I created 2 buttons. The first button is the button I will use to activate the CodedUI code (this is the spec flow code in our real life scenario) and one other button that CodedUI code will click after I clicked the “click Hello Automated” button. I called the second button “Hello”.

Here is a screenshot of the applciation:

image

Now to get started, You need to add quite some references to assemblies to get CodedUI to run within your application. Therese references don’t seem all to obvious, but without them, your code will crash at runtime with Assembly type load exceptions.

You need to reference the following set of assemblies: (I have them in the format, so you can copy and paste them straight into your csproj file, after unloading it in the ide, to save you some time)

<Reference Include=”Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL” />
<Reference Include=”Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL” />
<Reference Include=”Microsoft.VisualStudio.QualityTools.WindowsStore.CodedUITestFramework, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL” />
<Reference Include=”Microsoft.VisualStudio.TestTools.UITest.Extension, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL” />
<Reference Include=”Microsoft.VisualStudio.TestTools.UITest.Framework“>
<HintPath>..\..\..\..\..\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PrivateAssemblies\Microsoft.VisualStudio.TestTools.UITest.Framework.dll</HintPath>
</Reference>
<Reference Include=”Microsoft.VisualStudio.TestTools.UITest.Playback“>
<HintPath>..\..\..\..\..\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PrivateAssemblies\Microsoft.VisualStudio.TestTools.UITest.Playback.dll</HintPath>
</Reference>
<Reference Include=”Microsoft.VisualStudio.TestTools.UITest.WindowsStoreUtility“>
<HintPath>..\..\..\..\..\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PrivateAssemblies\Microsoft.VisualStudio.TestTools.UITest.WindowsStoreUtility.dll</HintPath>
</Reference>
<Reference Include=”Microsoft.VisualStudio.TestTools.UITesting, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL” />

Next we need to write some code to click the button.Since we are not part of a classic CodedUI test, we need to initialize the payback engine ourselves first. so therefore we start with a call the Playback.Initialize().

Next we need to find the application under test, and this is our own process. Therefore I used the environment class Process to get the current running process information and that we can pass to the ApplicationUnderTest and attach to it. With this we now have a search scope for all CodedUi controls we want to find. Then you create a WpfButton and pass it the search criteria how to find the button. I have done this by searching on the x:name property of the xaml control. Here is the code:

Now you normally would just call this method from the event handler of our button. But if you do this the code will not run properly. this has to do with the fact that the event handler of the button click that would activate our code, runs on the UI thread. CodedUI will create a “search” control that will then search for the UI control on the screen, but on the same UI thread that needs to receive the input of the mouse click.  So we will block the UI thread until we would be done, blocking it from getting the codedUI input.

There is an easy fix for that, and that is to move the code to a background thread, enabling the UI thread to receive input.

So the event handler now looks as follows:

The final result now looks as follows:

Untitled

You see me click the activation button, then coded UI will click the Hello Button and then the Message box pops up.

All in all not the most complicated code you have ever seen I think, but still there where at least two things that where less obvious I think. First to initialize the playback engine and second the threading issue. (besides finding all the assemblies you need to reference, which is an annoying and time consuming task :-))

Hope this helps!

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

Tweaks after setting up a Mac Book Pro with boot camp as a Windows machine

For a couple of weeks I have a Mac Book Pro as my main workstation. For my work I need to spend quite some time in Windows and only for my cross platform mobile app development with Xamarin I need the Mac Book, so I can develop iOS applications. The past weeks I have been using Parallels virtualization, but when you use it for Windows phone development, you need to enable nested virtualization and then unfortunately the machine has strange lags and is sometimes extremely slow.

One thing I wanted to try out today was to setup Parallels in a different way, using a boot camp partition. Here you install windows on your bare metal using the Apple provided boot camp options.

I must say, installing with boot camp was overall a smooth install, no strange issues I needed to solve. Just a couple of tweaks that I summarize below:

Change the scroll direction on the boot camp windows machine

Only a few things I got used to the past weeks on the Mac Book I really love where the way the track pad works. One thing is natural scrolling, which I got used to in such a way that I could not work with the reverse scroll anymore on windows.

To fix the scroll directions, there is a simple registry key you can set.

Here is the Power Shell command you can use (in admin mode!)

Get-ItemProperty HKLM:\SYSTEM\CurrentControlSet\Enum\HID\*\*\Device` Parameters FlipFlopWheel -EA 0 | ForEach-Object { Set-ItemProperty $_.PSPath FlipFlopWheel 1 }

After rebooting, your windows will scroll in natural scroll mode, just as on Mac OS

Slowing down the scroll speed on the boot camp windows machine

The track pad now works, but the scroll speed is way to fast, at least for me it is. The way to slow down scrolling on the track pad is by changing the mouse settings in the control panel.   Go to the control panel, type in the search keyword, mouse and then select “change mouse settings”

Here you select the tab wheel and set the number of lines to a lower value. for me 1 works best as shown in the screenshot:

image

Enabling Hardware virtualization on your boot camp windows machine

This one is in the category don’t ask how it works, but it works.

If you first boot into your boot camp partition, you can not enable Hyper-V. reason is that the hardware virtualization is disabled and you normally need to enable this in the PC bios. In this case you need to take the following steps:

restart your machine and use the option key at startup to select the OS-X partition and boot into the Mac OS again. there go to the System preferences and then to the Startup disk option. Then you select your boot camp partition and click reboot.

Now windows starts again, and if you look at the processor information, you can now see the hardware virtualization is enabled.

image

With this fix I was able to enable Hyper-V, and enable Visual studio to install the phone emulator image and start working on Windows Phone projects again

Hope this helps.

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

Solving intermittent DNS issues
on your custom Azure
development domain

The past few weeks I have been working on my own development domain, hosted in windows azure. There is a nice step by step plan available online on how to set up a network that can host your own development machines and your own domain controller, so you can use all these machines as if they run in one domain. (http://azure.microsoft.com/en-us/documentation/articles/active-directory-new-forest-virtual-machine/)

Setting up such a domain is especially useful when you want to leverage e.g. TFS and the test tools, since these tools work best if you work in a domain joined environment.

After setting this up, all worked great and I have used it for several demos and just some average development work I do.

My network looks more or less as follows:

image

But all of a sudden last week, I ran into the problem that on any of my development machines (the test clients) could not connect to the internet anymore. Sometimes an address was reachable and all of a sudden a couple of seconds it was not possible to connect. This intermitted problem was really problematic getting any real work done these days especially when you are running tests against websites on the internet Smile

So trying to resolve the problem, I first looked at my network setup and especially DNS. Reason for me to start there is that the IE browser constantly showed the dreaded connection failed and telling me I did not have an internet connection. If you look a bit deeper, you could see it was a name resolution issue, so I started a command line and used nslookup to try to resolve names.

the results for such an lookup looked as follows:

C:\Users\marcelv>nslookup
> google.com
Server:  UnKnown
Address:  10.0.0.5

DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
DNS request timed out.
timeout was 2 seconds.
*** Request to UnKnown timed-out
>

So this confirmed to me I had a DNS problem, question next was how to resolve it. With the custom domain setup, I configured a virtual private network with the address range 10.0.0.x and since I wanted to host my own Active Directory  and you can see by the response it did pick up my DNS server. So the problem must be at the DNS server itself.

Then I looked at the DNS configuration and since it was fully functioning before, it must have something to do with a DNS setting that might have gone stale in the Azure network topology.

One of the things that is different to on premise AD is the fact that your AD machine still has an DHCP obtained address. I did pin it to a fixed address using the PowerShell command described, but still you get this address from a DHCP server.

Now looking at the configuration, I discovered that the configured forwarder, that takes care of all requests outside the local domain, was unable to resolve.

image

After posting a request on the azure forms, to get some help, I got a response that was very helpful. It pointed me to the following article: http://windowsitpro.com/windows-azure/solve-dc-name-resolution-problems-azure

This was exactly my issue, so by removing the forwarder, DNS will fall back to the root hints. Now all requests can be resolved straight away and my development machines can happily browse the web again Smile

I Assume this problem is caused by the fact that once I created this AD machine and promoted it to the AD and DNS server, it just added the DHCP server that it got it’s lease from as the default forwarder. Since I might have been moved around the datacenter during maintenance or perhaps even failover, it is very well possible this server does not exist anymore. By just removing it from the list, I have a more sustainable solution, since it will not be affected anymore by such a move in the future.

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

How to faster close a browser tab when using CodedUI

Last week I got contacted via twitter (@marcelv) with the a question regarding my Pluralsight CodedUI course and a previous blog post that shows how to use CodedUI to test applications that use multiple tabs in the browser.

This particular question was, how can I close the browser tab faster since using the BrowserWindow.Close() function takes a lot of time.

To give some more context, in this blog post you can read how to access a newly created browser tab in your test.

Now when you have found the new tab, you have a reference to that tab. This reference is a BrowserWindow class. This class has the close method on it, but if you call the method to close the tab, this can take quite some time. On my machine times varied between 8 and 14 seconds before the tab was closed. If you have an application that has a lot of tabs that you need to test, you can imagine test times add up quickly.

So how does the code look like if I close the tab using the BrowserWindow class

 

So what is the alternative, and how can I close the browser window in just a matter of milliseconds?

Well this is the area that CodedUI shines in my opinion (Besides the point that the close takes way to much time, which is of course not cool at all!). With CodedUI I can very easily switch UI technologies while still using the API. What I mean by that is that in stead of using the browser window and HTML controls that use the web technology driver to find element, I can just switch to using Windows Controls, that uses MSAA to access controls on a windows desktop,  and manipulate those in order to achieve my goal.

In this particular case, you need to find the tab strip in the browser window and in the tab strip find the tab you want to close. If you run your test in internet explorer, the tab contains a close button that you can then find and click.

The following lines of code achieve this goal (assuming the tab has the title “New Tab” as with the example in my previous post):

 

So what is the difference in speed her?

Well using the BrowserWindow class it took between 8 – 14 seconds to close the tab, with the alternative approach using windows controls it takes about 500ms.

there is a downside to this approach that you need to be aware of, when you run multi browser tests with the selenium plugin that MS provides via the visual studio code gallery, you need to be aware of the fact that using the BrowserWindow class works cross browser and using the alternative only works for the browser you design the close for. So if I want to achieve the same on Chrome or Firefox, then I need to figure out which windows controls they use for their tabs and how I can click the close button there as well.

Hope this helps

Marcel

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

Creating a detailed test report
showing results of test case steps

For a while I have been working on a fairly large implementation of Team Foundation Server. During this implementation we phase out several silo based systems, like rational tools and HPQC. Moving from HPQC to MTM and TFS has been rather easy if you understand that the two systems just have a complete other way of treating test cases and test runs. To Satisfy some end user data needs we needed to create a custom report that is similar to the reports that can be created with HPQC out of the box. This is the test detail report, which I added here below to see what it is about. clip_image002 As you can see it shows not only the test suite outcome, but also the outcome of the test case and the detailed steps and where things failed. Creating a report as above, apeared to be more challanging then I thought at first, but that has to do with the fact that the standard solution we have, don’t have all the data we need. Work item queries, don’t have test case results. The data warehouse contains data to the test case run level, but not the step details. So how can we fix this? Apparently my fellow MVP Tarun Arora (http://geekswithblogs.net/TarunArora) had an excellent post (http://geekswithblogs.net/TarunArora/archive/2011/10/02/mtm-testing-scorecard-using-tfs-api.aspx) that got me out of the gate towards a solution. What he did is just create a simple windows forms application that uses the TFS API to get to all the data. So while less ideal, we can have a solution, but unfortunately not one that integrates out of the box. So I picked up the sample code from Tarun and altered it in such a way that it would give full details on test runs and render them using an embedded report viewer control. The only alternative to get this data is using the TFS object model. In the object model I can get to all the details I need, by using the ITestManagementService. So when you want to get to the test details, you first need to understand how the relationships are set up in the API. So first let start with understanding the basics, and then dive into the solution.

1.1 TFS and MTM ITestManagementService data

To get access to the service we use the standard pattern of connecting to TFS and then requesting the service we need.

Now that I can get to the service, we need to walk the hierarchy. We have a Team Project and a Team project can have multiple test plans. A Plan has the notion of Test Suites and a suite can be either a static suite, a Query based suite or a Requirements Based Suite. Now the tricky part is that a test case is a work item that can be associated with multiple test suites and a test suite can contain multiple test cases. The following diagram shows the relationships.   image Now this model is the logical setup when you look at how you create a test plan. The moment you want to execute a test, we need to add some additional things to the mix. Test runs are a mixture of Test suites that contain test cases and based on the selected configurations we want to run for. So if you start to look at test results, the data structure is a bit more complex. So For our report to succeed, we need to retrieve the following data:

  • For a selected team project ( a dropdown selection in the tool) and a selected Test Plan (Also a drop down selection) we are going to retrieve the test suites.
  • Then for each suite we need to get the test cases and for each test case we need the test results.
  • These test results can be found by using a concept call “Test Points”. A Test Point represents a Test Case to be run against a Test Configuration in a Test Suite.

The results become even more complex, since we can execute a test that contains test cases with parameters (using the @ symbol to denote the parameters in the test steps) and this will result in multiple iterations of the same test steps. The below picture shows the result structure we need to use to get the right data: image So as you can see it involves quite some iterations to get to the final data we want. What I have done is used the solution Tarun build and extended it to support my needs. I had to alter some of the functions that get the test suite hierarchy and the actual function that retrieves the data and creates dataset that can be fed into the report viewer control. To give you an example of the results, I have a couple of screenshots to show you how it looks. You first create a test plan in MTM, like this: clip_image008 You can see I have a test plan with several test cases in there and I ran a set of manual tests so all my tests have some test results associated. Now when you start the report tool, you need to connect to your TFS server image After you connect, you select the team project and the plan you want to view the results from. image When you selected them, click the generate report button, and it will get the data from TFS image Next you can see the detailed report with all test suites, test cases per suite, per test case the iterations and per iteration the test steps and the result of each step. As a bonus, I added also the capability that when you are using parameters in a test case and have multiple iterations on a test run, you get the parameter values that were used in a run as well. Smile clip_image016 I thought it might be useful for others to use as well, so I placed this work on codeplex. Now everyone can contribute. The codebase is not the greatest code in the world, but it gets the job done for me at the moment. Feel free to make changes and tidy things up as you use the tool yourself. I love contributions Smile You can find the project here: https://tfstestreport.codeplex.com/ If you want to join, please let me know and provide me your login name at codeplex when requesting. Hope this helps, Marcel

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

Older posts Newer posts

© 2018 Fluent Bytes

Theme by Anders NorenUp ↑