Fluent Bytes

"The only source of knowledge is experience" - Albert Einstein-

Execute and publish xUnit Tests results with .NET Core and VSTS

At the moment of writing you can not use the standard Visual studio tests task to run your xUnit unit tests on DotNetCore 1.0 and then publish the results to VSTS.

You need some different steps to get DotnetCore compiled and run your unit tests in a VSTS build. Here are the steps to make it work:

Pick the standard Visual studio template for your build:

image

In this default build workflow first add a command line to run a dotnet restore command so your build of your solution will succeed. do this right after the nuget restore step:

image

In the command line options set the tool to : dotnet and the command line argument to: restore

image

Now our solution should build.  The next step is to replace the Test assemblies task since in dotnet core you run your unit tests also using the dotnet command line. So we replace this task with a new command line task and provide it the following arguments:

test $(Build.Repository.LocalPath)\<locationoftestproject> –no-build -xml testresults.xml

Of course here you replace <locationoftestproject> with the correct location of the project containing your unit tests. In my case I was using the MVC music store sample project (which can be found here: https://github.com/aspnet/MusicStore )and for this I needed to replace the test location to: test\MusicStore.Test

Note that the additional arguments instruct the framework not to build the solution again, since we already have done this and would be a waste of time. The –xml option provides the name of the xml test results file that we want as output so we can publish the results to VSTS.

image

Now we have run the tests, so the final step is to publish the results to VSTS. For this we use the Publish test results task

image

And for this task we need to provide the name of the file it needs to publish. this is of course the name of the file we specified as the –xml output. We also need to specify that the results file contains test results in the xUnit format. Default it is set to use the JUnit format. But this is a simple select from the dropdown test Result Format.

So the final settings for the Publish task should look like this:

image

Now we can run our build and you will see the following results appear:

image

As you can see our tests are run and nicely reported. In this screenshot I switched to the tests tab in the results views that are available for a build and changed the outcome to show all results. Here you can see all de details of every test you have, when they have started failing, and how long they run.

As you can see all integrates nicely, only not with our default Visual Studio test task. I assume we might see this integrated in the future when we get new releases of the build tools for DotNet Core, but for now this is a pretty nice and clean solution to get things up and running.

Enjoy!

Marcel

CTO at Xpirit, Microsoft Regional Director, Visual studio ALM MVP, Speaker, Pluralsight Author and IT Architect Consultant

12 Comments

  1. Thanks for this amazing post! It was really helpful.

  2. For some reason I cannot get “–no-build” to work. It fails with “No executable found matching command “dotnet-test-xunit” no matter what I try.
    Leaving it out means an extra redundant build but at least the tests will be run fine. I have no idea why it fails, maybe one of many small quirks regarding .NET Core that will sort itself out as it matures.

    Great article though!

  3. Hi, i am trying to do the same, but, finally, i ve got this:
    No executable found matching command “dotnet-test-xunit”

    So, i was wondoring if there something else to add as step. How to know wich dotnet version the build is supported?
    I am using .NET 4.6.1 and AspNetCore

  4. Thank you Marcel! This saved me a lot of time!

  5. Hi all,

    Thanks Marcel for this great article.
    This line will fail though because “–no-build” was transformed to a strange long dash (word or something?) .

    So “test $(Build.Repository.LocalPath)\ –no-build -xml testresults.xml”, should be replaced with “test $(Build.Repository.LocalPath)\ –no-build -xml testresults.xml” (like in the screenshot).

    Thanks for your post, you saved me alot of time!

  6. For some reason I get the test results with the Total number of tests passed/failed, Test Duration in the banner, but the list of individual tests does not populate. The tests list control just says “This build doesn’t have test results for the outcome you’ve selected.”

  7. Cedric Tallichet

    March 14, 2017 at 12:49 pm

    For the SDK 1.0.1, the dotnet test argument -xml does not exists anymore. It should be replaced by –logger, for instance:

    dotnet test –no-build –logger “trx;LogFileName=testresult.xml”

    The output is not exactly the same but it is read correctly by publish test results.

Leave a Reply

Your email address will not be published.

*

© 2017 Fluent Bytes

Theme by Anders NorenUp ↑