(testng多个class文件执行时混乱，不是等一个class内的所有methods执行完再去执行下一个class内的内容)问题的解决

问题描述如下：

 

We use TestNG and Selenium WebDriver to test our web application.

Now our problem is that we often have several tests that need to run in a certain order, e.g.:

• login to application
• enter some data
• edit the data
• check that it's displayed correctly

Now obviously these tests need to run in that precise order.

At the same time, we have many other tests which are totally independent from the list of tests above.

So we'd like to be able to somehow put tests into "groups" (not necessarily groups in the TestNG sense), and then run them such that:

• tests inside one "group" always run together and in the same order
• but different test "groups" as a whole can run in any order

The second point is important, because we want to avoid dependencies between tests in different groups (so different test "groups" can be used and developed independently).

Is there a way to achieve this using TestNG?

Solutions we tried

• At first we just put tests that belong together into one class, and used dependsOnMethods to make them run in the right order. This used to work in TestNG V5, but in V6 TestNG will sometimes interleave tests from different classes (while respecting the ordering imposed by dependsOnMethods). There does not seem to be a way to tell TestNG "Always run tests from one class together".
• We considered writing a method interceptor. However, this has the disadvantage that running tests from inside an IDE becomes more difficult (because directly invoking a test on a class would not use the interceptor). Also, tests using dependsOnMethods cannot be ordered by the interceptor, so we'd have to stop using that. We'd probably have to create our own annotation to specify ordering, and we'd like to use standard TestNG features as far as possible.
• The TestNG docs propose using preserve-order to order tests. That looks promising, but only works if you list every test method separately, which seems redundant and hard to maintain.

Is there a better way to achieve this?

I am also open for any other suggestions on how to handle tests that build on each other, without having to impose a total order on all tests.

PS

alanning's answer points out that we could simply keep all tests independent by doing the necessary setup inside each test. That is in principle a good idea (and some tests do this), however sometimes we need to test a complete workflow, with each step depending on all previous steps (as in my example). To do that with "independent" tests would mean running the same multi-step setup over and over, and that would make our already slow tests even slower. Instead of three tests doing:

• Test 1: login to application
• Test 2: enter some data
• Test 3: edit the data

we would get

• Test 1: login to application
• Test 2: login to application, enter some data
• Test 3: login to application, enter some data, edit the data etc.

In addition to needlessly increasing testing time, this also feels unnatural - it should be possible to model a workflow as a series of tests.

If there's no other way, this is probably how we'll do it, but we are looking for a better solution, without repeating the same setup calls.

 

 

 

 

 

 

 

 

 

 

he situation can easily be reproduced using the code snippet

below.

I'd be really glad if the ordering could natively be turned to "per class
atomicity"

 

------------ ClassA.java --------------
package test;

import org.testng.annotations.Test;

public class ClassA {
        
   @Test
   public void testA1()
   {
      System.out.println ("testA1");
   }
   
   @Test
   public void testA2()
   {
      System.out.println ("testA2");
   }
   
   @Test(dependsOnMethods={"testA1","testA2"})
   public void testA3()
   {
      System.out.println ("testA3");
   }   
}

------------ ClassB.java --------------
package test;

import org.testng.annotations.Test;

public class ClassB {
        
   @Test
   public void testB1()
   {
      System.out.println ("testB1");
   }
   
   @Test
   public void testB2()
   {
      System.out.println ("testB2");
   }
   
   @Test
   public void testB3()
   {
      System.out.println ("testB3");
   }
}

------------ testng.xml --------------
<?xml version="1.0" encoding="UTF-8"?>

<suite name="TestSuite" parallel="none">
  <test name="MyTest">    
    <classes>
      <class name="test.ClassA"/>
      <class name="test.ClassB"/>
    </classes>  
  </test>
</suite>

 

------ Output -------
[TestNG] Running:
 
/usr/users/arno/local_home/workspace/playground/src/test/testng_dependsOnOrdering.xml

testA1
testA2
testB1
testB3
testB2
testA3

I have like 30 test-classes each with 4 or 5 tests and have 
dependsOnMethod on them. testng.xml would be unmanagable if I have to 
add <test>  for each one of them.

 

 

 

 

 

 

 

目前来说的解决办法：
the test-runner _interleaves_ different tests from different classes!
Tests in the same <test> tag will always be run interleaved, if you want them separate, you should probably put them in different <test> tags...

 

 

不知道testng后续会不会优化，这个的确很annoying