In Part 1 of Why Dedicated Software Development Testing Matters, key reasons for why comprehensive Testing is so critical to producing high quality software development projects were presented.The post also included descriptions of the first two phases of the Quality Control Structure and Testing Flow and highlighted key factors in testing execution. If you missed the post, you can read it here.
In Part 2, Phase Three of the Quality Control Structure and Testing Flow is introduced plus a listing of critical must-have testing skills, and descriptions of the most common types of software testing.
Testing teams must always address these three questions:
To answer these questions, the testing team should employ the use of smart metrics. These metrics can include testing statistics such as: number of attempted cases, passed/failed and product statistics which include the number of open defects, defects by status, defect density per lines of code, and so forth.
Metrics are a powerful tool for testing project management. FLS typically recommends using a default template for test product metrics, which can be customized according to the specific project.
When it comes to readiness reporting, it is not enough to simply list covered features and related defects. While it is possible to deliver working software with major bugs; what matters most is to ensure the Product Owner understands the defects so they can make an informed release decision. First Line includes an Executive Summary in the Readiness Report with recommendations for the development team and client about what is critical to address prior to the product release.
Testers need to understand the project goal, the business itself and the requirements for the product being built. They should have some BA skills, or work with BAs and the Project Manager as a team, to select the appropriate quality criteria.
In an Agile environment, testers analyze the scope together with the development team plus a BA, Product Owner (PO), or a PO deputy, to ensure the scope is transparent, the testing goals are understood and the team is on the same page. With this approach, any questions can be addressed immediately. Finally, the team agrees on the acceptance criteria and works out the common goals.
Testers should also have a solid technical background to ensure they understand the project technology and architecture – at least on a high level. Testers who meet the criteria will be able to tell which part of the product is being developed and what is not considered a part of the development project. A skilled tester will also be able to identify the potentially fragile areas and accordingly, place a greater focus on them.
A range of testing types may be used to ensure product conformance with requirements. Depending on the Systems Development Lifecycle (SDLC) and project goal, testing can be manual or automated. The most common types of testing are:
Static Testing and Sanity Testing – used primarily to verify that requirements are clear and testable.
Functional Testing – Functionality is at the core of the application. Various practices are used to make functional testing effective. For example, some projects use a Transition procedure which enforces communication between developers and testers to help identify the most obvious issues early on. Agile projects use Sprint Planning during which developers and testers discuss functionality before starting development, clarify all details and ensure final transparency for expected results.
Load Testing – This non-functional testing is focused on product behavior under various load levels. For example, we use the Load Impact tool to create scripts which emulate large numbers of simultaneous users opening different web pages. The groups are distributed to a number of geo-locations and user behavior is scripted using Lua language. The testers create scripts, define the load level and schedule the scripts. While the scripts are running, the team monitors the back-end using server agents and collects key metrics such as CPU usage, Memory usage, network etc.
UI and UX Testing – With the vast number of mobile devices in use and frequently changing browser specifications, this testing is important for identifying UI issues that will negatively impact the user experience.
Integration and System Testing – This type of testing is extremely important for complex projects. Once the components are tested, the connectivity between the components needs to be verified. System testing means testing software as a whole, not part by part.
Regression Testing – Regression testing is more about when and what to test rather than how. There are times when testers need to verify that the product is free from degradation, which can be introduced as a side-effect of recent changes.
Smoke Test – Similar to Regression Testing, it is not about how, but about “what”. The Smoke test is a small scenario aimed at checking if the application is ready for deeper testing and is often also used for products with frequent releases – after deep functional testing – as the final activity to make a Go-No Go decision. A Smoke test needs to be focused on what is most important to the specific development stage.
Mobile testing - Testing on mobile devices is a must-have for most, if not all, development projects these days. Whether it is a native app or a site with mobile support, it is important for testers to keep in mind the list of mobile-specific cases such as: screen rotation, low Wifi, UI guidelines etc.
It is critical to use the right tools for the best testing results. The following list provides examples of testing tools used for specific types of testing by First Line testing experts:
Test Case Management - TestLink, Zephyr, a Jira plugin, MS MTM
Bug Tracker - Jira, TFS, Redmine
UI Testing and Script Verification - Firebug, YSlow
Load Testing - Load Impact, JMeter
Functional Test Automation - Selenium, Ranorex, TestComplete, and Coded UI Tests (CUITs) for Microsoft
There’s no doubt Software Development Testing is a comprehensive topic and one that must be given adequate attention and consideration before launching any development project. We hope this post has been convincing about how critical Testing is to the success of software development projects.
Whether you have a complete software development project in the works or are seeking a project partner with Testing expertise, we invite you to contact First Line for an exploratory discussion about your development and/or testing requirements.