AgileWay Test Automation Formula, Part 1

Recently, a former colleague asked me for a successful test automation formula. This happened before; my default answer was to refer him to my book: "Practical Web Test Automation", and advise him to do the exercises, and then apply them to work. I believed that the success of test automation depends on many attributes: knowledge, willingness to learn and be ready to change, test frameworks/tools, management support, .. etc. More importantly, do it hands-on from Day 1, and every workday onwards. There is no silver bullet.

When I put more thought into it this time, while I still hold the belief that successful test automation has many factors, I believe that there is a good pattern people can follow. A formula, based on my understanding, is a combination of components and processes that has been proven and others can copy and apply to use quickly. The universal applicability makes sense as our target (e.g. web apps) has not changed much.

An important attribute of the formula is completeness. To give an opposite example, I heard this in a meeting at a large financial company: "The DevOps team has been working on trying to run Micro Focus UFT tests in TeamCity for over 9 months. Can someone give us an estimation of the completion date?". I didn't remember the reply (from the DevOps team leader) to this as I was very shocked by the question at that moment. Apparently, the decision of the CI Server has not taken the UI tests into consideration! This means the people behind do not understand test automation or CI/CD at all, and they are still there.

By following a good formula, inexperienced engineers may avoid common mistakes, such as using Cypress, Gherkins, or JavaScript (as the scripting language).

Projects often make wrong choices on technologies for Test Automation

If a software project chose the wrong technology, it was usually very hard (almost impossible) to correct. Reasons are:

• Management generally won't admit the wrong decision and take responsibility.
• The tech lead will usually defend their wrong approach, and worse, sabotage correction efforts.
• Most IT engineers have never seen a single successful implementation of automated end-to-end testing.

Therefore, a good formula might not guarantee you success, it will prevent costly mistakes and keep you on the right track. Some talented and passionate engineers might be able to pull that off, or wise managers may seek help from a real Test Automation coach.

AgileWay Test Automation Formula

I have a good track record (10 years+) of doing successful Automated UI Testing which is widely considered difficult. I have used the same technology /tools /practices successfully in many projects. I name it "AgileWay Test Automation Formula":

• Automation Framework: Selenium WebDriver or Appium (raw)
• Scripting Language: Ruby
• Test Syntax Framework: RSpec
• Source Control: Git
• Testing Tool: TestWise IDE
• Continuous Testing Server: BuildWise, Parallel Testing with BuildWise Agents

Note: the above are 100% freedom, open-source, widely-used frameworks. Even for the tools, you don't need to pay a cent to get started or forever (just like Zoom, you may use it in free mode forever).

TestWise and BuildWise (in italics) are my choice of tools (disclaimer: I created them, and both were received well internationally), which means they are optional. It is the tech leads' responsibility to select high-productivity tools. A rule of thumb: manual testers can set up and run tests ~15 mins; CT server can be set up with parallel test execution < 1 day.

Over the last 12 years, I have successfully applied this formula to numerous projects with instant success: at least got one key automated test (not simple login) run in the CT server on the first day. (then added more tests, maintained all, and ran them daily). How could I accomplish that? Easy, because our target (the technologies behind web apps: HTML, JS, and CSS) changed little over the last decade. This means that my formula has worked successfully at Company A, just re-apply it at Company B.

Please note, you have total FREEDOM with this formula, and it can be set up quickly (under 15 mins for developing/executing automated test scripts, another 10 mins for executing them in CT Server sequentially, parallel execution will take a few more hours as setup of agent machines are required).

--

In Part 2, Automation Frameworks and Scripting Language.

In Part 3, Test Syntax Framework and Continuous Testing Server

The original article was published on my Medium Blog on 2021-03-31