How Does MSTEST/Visual Studio 2008 Team Test Decide Test Method Execution Order?

Question

I was under the impression that the test methods in a unit test class would be executed in the order that they appear in the class file. Apparently this is not true. It also doesn't appear to be purely based off of alphabetical order either. How does MSTEST decide execution order?

EDIT: I was able to track down the answer after digging a bit. See below.

Answer 1

I was able to track down the answer.

According to Microsoft employee Guillermo Serrato:

MSTest executes all tests synchronously, the order is nondeterministic

Answer 2

This is actually simply to do: use this link How to: Create an Ordered Test

Answer 3

Sorting in Test View, or test list editor causes them to appear like you have control -- it's just a by product of the implementation. We make no attempt to actually execute them in a specific order (We've gone back and forth on the "allow order", and "randomly order").

If you really need order, that is what ordered tests are for. These are available in all editions where Unit Test is available -- either use the Test/New Test menu, or right click the test project and create a "ordered test"

Answer 4

It's very simple. I am using this

C:>mstest /testcontainer:C:\MyTest.dll /resultsfile:C:\MyTestresults.xml

The logic is here that by default mstest gives you a .trx file. So in the command against [/resultfile:] option write the name of file by giveing .xml extension. So I used /resultsfile:C:\MyTestresults.xml instead of /resultsfile:C:\MyTestresults.trx

This works fine for me. Let me know if it works for you.

Answer 5

In my experiences (I was little hard) I tried a lot of thinks about the change order of the method execution orders in the MStest project. I'm going to try describe the steps.

1. You should ensure delete the default .testrunconfig and .vsmdi files. These files createdby Visual Studio when the creating test project.

2. Close the Visual Studio instance and reopen it.

3. When the Visual Studio opened you should open the Test List Edior then Create your own list under the Lists of tests section. After you create a new test list it should be empty.

4. Go to All Loaded tests section then drag any test onto newly created test section with your choised order.

Then ensure your last created test section checked and you can press the run test and voala it's running with your defined order.

I don't know this might bug or some thing like that, but it magically working. We tested this method a couple of time with my colleague.

Good lock

Answer 6

Problem: What I want to do is to run only the web tests that are included into my ordered test. I'm using an ordered test just for that purpose, so I can run my web tests in the specific order I need them to run in. I have lots of web tests, but my ordered test contains a sub-set of those web tests, which I would like to run. Solution: An ordered test is used to run a set of tests in a certain order. Create an Orderer test list in Visual Studio/test project. From test view window (Test | Windows | Test View), select the OrderedTest and run it from there (you can right-click it and select "Run Selection"). The orderedtest will automatically run all your webtests that are included in it.

You will the the results in the test results window (Test | Windows | Test Results); double click the OrderedTest, and you will see detailed results for everyone of your webtests there. You can double click each webtest, and you will be able to see detailed results for each of them.

Answer 7

There are many ways to order the tests in VS. Use the test view and the add extra columns and order. I use VSMDI files and this runs them in order specified therein.

Answer 8

It shouldn't matter what order they run in ... if your tests depend on another test running first, your tests are flawed :-)