Test Automation and Continuous Testing Competition Week. Part 2: Assessment
A Practical and Fun Way to Create a Successful Test Automation and Continuous Testing Formula in Your Company
Continue from Part 1.
6. Judge objectively and solely based on the results.
The judgment criteria:
- Completion of the target test suite.
- Correctness.
- Execution speed.
- Detection rate for the known effects
- Execution reliability in CI/CT server. The overall pass/fail rate is not a good criterion. A better way is to check whether a green build can be obtained (pass all tests) at the end of the day.
- The quality of test scripts. A simple way to judge is to check whether the business analyst or manual tester understands the test scripts.
- The finishing time. On the comparable results, the team that finishes the test suite earlier wins.
Reward the winner!
The company surely needs to put into some investments for this competition. The cost, relative to the company’s gains or the money it would save, is minor.
Two outcomes resulted from the competition are:
1. A successful formula for test automation and CT have been found
A big win for the company is that the success will boost the confidence and grow the capability on this ‘proven’ track. I can share some experiences from the companies that do not have these open exercises.
- The tech leads and test manager decided on Cypress but found Cypress did not support iFrame a week later. (Cypress and some other JS-based automation frameworks have limitations, check out this article: Why Cypress Sucks for Real Test Automation? Part 2: Limitations)
- Cucumber was sold by ‘fake agile coaches’ to be used in testing. An agile coach boasted that he had done over 200 Cucumber end-to-end tests in several of his past projects. In reality, this test automation team (with the involvement of this ‘agile coach’) has never been able to deliver a successful test report over 12 Cucumber tests. The team was struggling with maintenance from Week 2 and onwards until these tests were aborted.
As we all know, after a decision was made, the management who made this decision (no matter right or wrong) often chose to defend it. That’s why so many companies tend to renew the licenses of testing tools though the companies do not actually use for real work.
Besides, the company gets a regression suite of 25 automated tests which are useful now.
2. No single team delivered satisfactory results
This is probably the likely outcome for most software companies.
Though it means the company does not have the capability, still a valuable finding for the company.
Then review and act based on the feedback:
- identify what approaches definitely won’t work, such as record-n-playback and the use of Gherkin syntax.
- identify the limitations of certain technologies, such as Cypress does not support two browser windows/tabs, …, etc.
- prevent many unnecessary (and costly) attempts from individual teams.
- improved awareness of infrastructure support for real Continuous Testing.
- improved awareness of test automation beyond QA. For example, Business analysts may want to run automated tests but are uncomfortable with the complexity of the tools.
- improved respect for CT engineers’ hard work
- help some engineers to give up fixation on the wrong framework/tools. Meetings won’t help, I tried. Once, most of the team members acknowledged I was a better Java programmer, but I couldn’t convince them to use Ruby to write automated tests after numerous discussions.
- …
If necessary, the company might do another competition event in a month's time. If it is still no luck, engage a real test automation and CT coach to help. Now the executive’s decision to seek external help from a real test automation coach might face much less obstruction.
Of course, the company can use the same competition process (with a shorter timeframe) to test whether a test automation coach candidate is fake or not.
---
- AgileWay Test Automation Formula
Relative readings:
About the Creator
Zhimin Zhan
Test automation & CT coach, author, speaker and award-winning software developer.
A top writer on Test Automation, with 150+ articles featured in leading software testing newsletters.
Keep reading
More stories from Zhimin Zhan and writers in Geeks and other communities.
Unit Testing Clarified, Part 2: FAQ
Continue from Part 1. Q & A 1. Why do the tech leads/managers/programmers use the ‘Unit Test’ term wrongly? Managers are excusable, but not for ‘tech leads’ and programmers. The short answer is that these tech leads have never written nor executed real unit tests. This is very concerning. We know that “Good programmers write unit tests”, and the T in TDD (Test-Driven-Development) is Unit Testing. If we have these fake/incompetent engineers as the tech leads, the project will be doomed.
By Zhimin Zhan2 years ago in Geeks
Book Review: "Human Monsters: A Horror Anthology" by Various Authors
Yes, it's horror again. I'm feeling kind of down and whenever I'm down I read horror because it makes me feel better. Have you heard about that study that states people who are in depression should watch and read horror in order to make themselves feel better? Yeah, I checked it out some time ago - you should too. Human Monsters is exactly what you think it is, it is about monsters who are human - those who lurk in the normal world, not in the shadows, not in the darkness - they stand right in front of us. They are us. Let's go through my favourite stories in the anthology...
By Annie Kapur22 days ago in Geeks
Federal Reinforcements and Rising Tension in Minneapolis
Tension in Minneapolis has risen sharply in recent days as the U.S. Department of Homeland Security (DHS) announced plans to deploy hundreds more immigration officers to Minnesota following the fatal shooting of a local woman by an Immigration and Customs Enforcement (ICE) agent. The move comes amid nationwide protests and deepening polarization over federal immigration enforcement efforts. The decision to send additional officers was publicly confirmed by DHS Secretary Kristi Noem, who said the reinforcements are meant to “ensure the safety” of ICE and Border Patrol personnel already in the Minneapolis–St. Paul area. Officials noted that more than 2,000 federal officers have already been stationed in the region as part of what the administration describes as its largest immigration enforcement operation ever. What Sparked the Deployment? The deployment follows the Jan. 7 fatal shooting of 37-year-old Renee Nicole Good by an ICE officer during an enforcement action in south Minneapolis. Good, a mother of three, was struck and killed by gunfire as she sat in her vehicle. Federal authorities have characterized the incident as a defensive action, saying the driver’s conduct endangered officers. However, local political leaders and community members strongly dispute that narrative. Minneapolis Mayor Jacob Frey and other officials have criticized federal accounts of what happened, pointing to bystander videos that they say contradict the claim that Good posed an active threat when she was shot. They have also condemned the broader federal immigration operation in the city, saying it has fostered fear and mistrust in local communities. The legal and investigative aftermath remains complicated. Minnesota authorities sought to investigate the shooting, but federal officials asserted exclusive jurisdiction, citing constitutional protections for federal officers. This has deepened frustration among state leaders who want greater transparency and local involvement in the inquiry. Protests and Public Reaction The shooting ignited protests in Minneapolis almost immediately and spurred demonstrations across the country. Over the weekend following the incident, more than 1,000 rallies were organized in cities nationwide, drawing tens of thousands of participants. Protesters have framed their actions as a call for accountability and change in how immigration policy is enforced. In Minneapolis, large crowds gathered near the site where Good was killed to march and call for an end to what they see as an aggressive federal presence. Many protesters have chanted slogans critical of ICE and the broader administration’s immigration agenda. Scenes from protests have been varied. In some locations, demonstrators have remained peaceful, holding signs and vigils in memory of Good. In others, confrontations between some protesters and law enforcement have occurred, with reports of officers deploying crowd control tactics. Supporters of the federal response argue that a strong enforcement posture is necessary to uphold U.S. immigration laws and to address alleged crime tied to illegal immigration. DHS officials also stressed that the officers being sent to Minnesota are there to protect their federal personnel and keep operations running despite protests. The Broader Context of Immigration Enforcement This episode is part of a much larger national debate over immigration policy and enforcement tactics. The Trump administration’s recent focus on interior enforcement — including deportations and raids — has been a defining pillar of its second term. Minneapolis, a city with a diverse population and sizable immigrant communities, has become a focal point for that strategy. Critics say that such heavy federal deployment in city neighborhoods undermines community trust and disproportionately affects non-violent immigrants and long-standing residents. Advocates for immigrant rights argue that punitive tactics fuel fear, discourage cooperation with local authorities, and erode civil liberties. Supporters counter that rigorous enforcement is essential for national security and upholding the rule of law. They argue that federal agents must be backed by sufficient resources, especially when facing hostile demonstrations or resistance. Voices from Minneapolis Local sentiment in Minneapolis remains deeply divided. Many residents expressed grief and shock over the death of Renee Good, while also expressing concern about the scale and nature of the federal response. A number of community members have organized vigils, public gatherings, and faith-based services to provide mutual support and to call for justice. Conversely, Minneapolis leaders have urged calm while also criticizing federal policies they claim escalate tensions. Mayor Frey has been vocal in his rejection of the federal narrative, insisting that the city deserves transparency and a fair process to understand what led to Good’s death. Looking Ahead As hundreds more immigration officers make their way to Minneapolis, the city and the nation find themselves at a critical juncture in the ongoing debate about immigration enforcement. The coming days will likely see continued protests, political clashes, and legal challenges as both sides defend their interpretations of events and advocate for their visions of public safety and justice. For now, Minneapolis stands on edge — a city caught between federal authority and local calls for accountability, grieving a woman’s death, and questioning how far immigration enforcement can go before it deepens divisions even further.
By Saboor Brohi 3 days ago in Geeks
Comments
There are no comments for this story
Be the first to respond and start the conversation.