At the moment of writing you can not use the standard Visual studio tests task to run your xUnit unit tests on DotNetCore 1.0 and then publish the results to VSTS.
You need some different steps to get DotnetCore compiled and run your unit tests in a VSTS build. Here are the steps to make it work:
Pick the standard Visual studio template for your build:
In this default build workflow first add a command line to run a dotnet restore command so your build of your solution will succeed. do this right after the nuget restore step:
In the command line options set the tool to : dotnet and the command line argument to: restore
Now our solution should build. The next step is to replace the Test assemblies task since in dotnet core you run your unit tests also using the dotnet command line. So we replace this task with a new command line task and provide it the following arguments:
test $(Build.Repository.LocalPath)\<locationoftestproject> –no-build -xml testresults.xml
Of course here you replace <locationoftestproject> with the correct location of the project containing your unit tests. In my case I was using the MVC music store sample project (which can be found here: https://github.com/aspnet/MusicStore )and for this I needed to replace the test location to: test\MusicStore.Test
Note that the additional arguments instruct the framework not to build the solution again, since we already have done this and would be a waste of time. The –xml option provides the name of the xml test results file that we want as output so we can publish the results to VSTS.
Now we have run the tests, so the final step is to publish the results to VSTS. For this we use the Publish test results task
And for this task we need to provide the name of the file it needs to publish. this is of course the name of the file we specified as the –xml output. We also need to specify that the results file contains test results in the xUnit format. Default it is set to use the JUnit format. But this is a simple select from the dropdown test Result Format.
So the final settings for the Publish task should look like this:
Now we can run our build and you will see the following results appear:
As you can see our tests are run and nicely reported. In this screenshot I switched to the tests tab in the results views that are available for a build and changed the outcome to show all results. Here you can see all de details of every test you have, when they have started failing, and how long they run.
As you can see all integrates nicely, only not with our default Visual Studio test task. I assume we might see this integrated in the future when we get new releases of the build tools for DotNet Core, but for now this is a pretty nice and clean solution to get things up and running.
Enjoy!
Marcel
Thanks for this amazing post! It was really helpful.
For some reason I cannot get “–no-build” to work. It fails with “No executable found matching command “dotnet-test-xunit” no matter what I try.
Leaving it out means an extra redundant build but at least the tests will be run fine. I have no idea why it fails, maybe one of many small quirks regarding .NET Core that will sort itself out as it matures.
Great article though!
Hi, i am trying to do the same, but, finally, i ve got this:
No executable found matching command “dotnet-test-xunit”
So, i was wondoring if there something else to add as step. How to know wich dotnet version the build is supported?
I am using .NET 4.6.1 and AspNetCore
Thank you Marcel! This saved me a lot of time!
Hi all,
Thanks Marcel for this great article.
This line will fail though because “–no-build” was transformed to a strange long dash (word or something?) .
So “test $(Build.Repository.LocalPath)\ –no-build -xml testresults.xml”, should be replaced with “test $(Build.Repository.LocalPath)\ –no-build -xml testresults.xml” (like in the screenshot).
Thanks for your post, you saved me alot of time!
Ah, apparently this is done within your wordpress text editor.
So that should be two dashes next to each other (“- -” but then without the space in between).
For some reason I get the test results with the Total number of tests passed/failed, Test Duration in the banner, but the list of individual tests does not populate. The tests list control just says “This build doesn’t have test results for the outcome you’ve selected.”
It is very well possible the tools just don’t work correct yet. The tools are quite in flux still.
You’ll have to change the filter for Outcome to “All” instead of the default value “Failed”.
For the SDK 1.0.1, the dotnet test argument -xml does not exists anymore. It should be replaced by –logger, for instance:
dotnet test –no-build –logger “trx;LogFileName=testresult.xml”
The output is not exactly the same but it is read correctly by publish test results.
thanks for the update!
Thanks, that was helpful, but actually it must be:
-l|–logger