It's been a long time since we did regular updates, and that's about to change. We've spent the past 18 months building and solidifying a team structure and a test platform, and it's now time to start talking about it. We intend to release some of our work under the GNU public license, and this background information should help you in making the decision to use it.

We will post a series of articles, which will describe our processes in detail. Then we'll round it out with several examples of how this framework solved most of the problems that typically plague test automation efforts. Comments and suggestions are welcome.

Here's how our team works:

We have four test library developers, each of whom "owns" various pieces of the AUT from a functional point of view. One person may own User admin and role-based security, another may own Workflow Definitions, etc
The library consists of several dozen VBS files, each of which more or less contains a single Class
Each Class represents some sort of functional object in the AUT, e.g. we have a class for "User", which contains members such as "UserID", "password", etc, and methods such as "commit" (like create), "delete", etc
When the AUT changes, our test scripts start failing, and it is the responsibility of one of the 4 core team members to file a defect (in the case of a product error), or update the corresponding library file (in the case of simple product change)
These scripts utilize the classes above in a simplified way--with no references to navigation. That's a key point in this discussion
We, along with the rest of the Quality Assurance team, write test scripts into a centrally located repository (Rally), each of which can be executed from a single server, independently from the test developer's workstation
So, here's an example of how a simple test script would look:

View as plain text
Visual Basic:
'Log in to the application as the super administrator
LoginProfile "SUPER"

'Create a User, and set a few key attributes
Set oUser = NewUser
oUser.sUserID = "inquisitor"
oUser.sGivenName = "Marcus"
oUser.sFamilyName = "Inquisitor"
oUser.sPassword = "qq"
oUser.commit

Logoff

'Log back in as this new user
Login "inquisitor", "qq"

'Check to make sure the page title greets the new user by name
ComparePageTitleWith "Welcome, Marcus Inquisitor"
Things to note:

No reference is made to a URL or test site location in this code--we maintain a list of test sites against which our tests can be executed, and this information is generated and passed to the LoginProfile method at run-time (via Environment variables)
The syntax is a little programm-y, but we've made attempts to make it as unintimidating as possible. It's still a hurdle for people with no programming background, but there aren't any loops or egregious indentation schemes or anything
At no point do you see the word "click" or "mouse over" or "in this field, enter "
The simple object syntax allows you to add and edit attributes without regard to the web form's tabbed interface or to the exact data type of the attribute--the data is mildly scrubbed within the object's commit method
Until the object's commit method is called, no work is being done against the AUT. This allows you to instantiate, for example, an existing object (one that's already in the AUT database) by setting its attributes, then performing another operation on it, such as delete or edit.
This setup allows us to create a number of entities very quickly, very simply, and have these entities interact with each other in ways that test the business logic. By abstracting the navigational steps, we lose the ability to do deeply integrated tests of the creation of these entities (checking input validation and the like), but we get a lot more out of our test automation this way, since it's more oriented toward the interaction of various entities with each other in ways that replicate user actions.

To put it another way, you can easily test how a user interacts with the system throughout the lifecycle, rather than just testing whether or not you can create a user.

This system is well-suited for teams who would like to use Business Process Testing (BPT) from HP, but can't afford it. That's why we wrote it. This allows us to write test cases from Business Analyst-provided Use Cases at a very high level. It doesn't catch some of the details, but it does catch the real problems that prohibit our users from interacting meaningfully with our system.

Next up I'll talk about our standard interface, more detail about how we use Rally, and the queueing system.