Test automation importance
With our change to agile, we have made an important paradigm shift. In the old days we would stick to our planned roadmap, and compromise either on delivery dates or software quality. In Agile software development we are willing to compromise on scope, but not on software quality or delivery dates. This means in short: no code goes out untested.
What tests to execute?
It is nearly impossible to test everything. There are so many types of tests, like: regression, story acceptance, developer, integration, usability, performance, security, upgrade, user acceptance etc. In our release planning, we must make a conscious decision which of those types we want to execute. Some of the test types may require additional experts, some of the tests may not be relevant, some may be just too costly.
The usual scrum tests Within our sprints, we tend to stick to regression tests, story acceptance tests, developer tests and integration tests. Developer tests are internal component tests, that test a component directly without the need of other components. Integration tests review the interaction between two or more components. Story acceptance tests verify the acceptance criteria. Regression tests are the story acceptance tests of previous sprints and may also include parts of the integration tests.
Which tests to automate?
For some tests it may not make sense to automate. Will the test ever be executed again? If not, then it will probably not pay off to automate it. Also consider the cost of test automation versus the time needed to test manually. Especially for UI tests, it may be cheaper to do manual tests. Drag and drop, and visual control of the result are very hard to automate. For normal webforms or low level service tests, automation is efficient.
What are the risks of not automating?
It is likely, that those tests that we decided not to automate, will not be executed in future sprints. So future sprints may break existing functionality without us knowing it. With less test coverage, we will get afraid to refactor our code. In the extreme case, we would end up with an application that we don't dare to touch. So it is very important to have automated regression tests with good coverage.
With our change to agile, we have made an important paradigm shift. In the old days we would stick to our planned roadmap, and compromise either on delivery dates or software quality. In Agile software development we are willing to compromise on scope, but not on software quality or delivery dates. This means in short: no code goes out untested.
What tests to execute?
It is nearly impossible to test everything. There are so many types of tests, like: regression, story acceptance, developer, integration, usability, performance, security, upgrade, user acceptance etc. In our release planning, we must make a conscious decision which of those types we want to execute. Some of the test types may require additional experts, some of the tests may not be relevant, some may be just too costly.
The usual scrum tests Within our sprints, we tend to stick to regression tests, story acceptance tests, developer tests and integration tests. Developer tests are internal component tests, that test a component directly without the need of other components. Integration tests review the interaction between two or more components. Story acceptance tests verify the acceptance criteria. Regression tests are the story acceptance tests of previous sprints and may also include parts of the integration tests.
Which tests to automate?
For some tests it may not make sense to automate. Will the test ever be executed again? If not, then it will probably not pay off to automate it. Also consider the cost of test automation versus the time needed to test manually. Especially for UI tests, it may be cheaper to do manual tests. Drag and drop, and visual control of the result are very hard to automate. For normal webforms or low level service tests, automation is efficient.
What are the risks of not automating?
It is likely, that those tests that we decided not to automate, will not be executed in future sprints. So future sprints may break existing functionality without us knowing it. With less test coverage, we will get afraid to refactor our code. In the extreme case, we would end up with an application that we don't dare to touch. So it is very important to have automated regression tests with good coverage.
Comments