What role does software testing play in the continuous delivery pipeline?

Imagine ordering something online and waiting weeks for delivery only to receive a defective product — cue refunds, replacements, and complaints, and rightly so. No one likes to wait forever only to find out the product is faulty.

The same goes for clients.

As a software developer, you want to be able to deliver a high-quality, functioning product to the client. You can always expect an odd bug or two, but the fewer you have, the better. This is why software testing is essential: It’s a quality-control safety net.

You can conduct your testing manually or automatically. Below, we’re going to go into the benefits of both, as well as their role in the continuous delivery (CD) pipeline. Here we go!

What are the benefits of software testing?

• Ensures the stability of new features
• Minimizes the chance of bugs making it through to the end user
• Gives developers a set of prescribed standards and goals they can work towards
• Saves money because a stable, bug-free program will require fewer checks and less maintenance

What’s the difference between manual testing and automated testing?

• Manual software testing is exactly what it sounds like: An actual person conducts all those checks to make sure the software does what it’s supposed to do. They click around and engage with the app or website to test its usability. The downsides with this are that it’s expensive, time-consuming, and prone to human error.
• Automated software testing is again, exactly what you think it is: Testing is done automatically. There are different checks you can do, from code checking to those that focus on user experience. This is part of continuous delivery — which we’ll go into more detail a little later.

Automated testing is far more reliable than manual testing, but the tests will only be as good as the test scripts themselves.

What is the continuous delivery pipeline?

Automated software testing is part of the continuous delivery pipeline, a process that describes certain automated steps in the software delivery process. The pipeline consists of continuous integration, continuous delivery, and continuous deployment.

Because certain checks are automated, the developers are relieved of a certain amount of pressure. It also reduces the chance of error and allows the development team to work with more confidence.

How does software testing fit into continuous delivery?

Continuous delivery is the middle stage of the pipeline: all the code that was created in the build stage is deployed to an automated testing environment. Once the code has passed all its tests, it’s ready for deployment.

There are different types of software testing stages that take place within continuous delivery, with each type assessing the code from a completely different angle. Here’s a closer look at each one.

Unit testing

This is the most basic level of testing. A team member enters units of standalone code into the test environment. Meanwhile, production code functions run using simulated inputs. You can then compare the result (or output) against what you initially expected. If it’s a match, the code passes. If not, the code fails and will need to be reworked.

Integration testing

This is when you will combine and test individual units against each other or as a group. The goal is to expose errors in the interaction between integrated units. Developers will use test drivers and stubs for these checks. It’s important to keep these tests separate from unit tests: developers will want to run quick unit tests during development before committing the code. If you start running integration tests at the same time, it’ll complicate and slow down the process.

Functional testing

Also known as ‘end-to-end’ testing, this check simulates the entire user experience. The tests perform things users regularly do while using the app or site, including filling out forms, clicking buttons, and going through checkouts. These tests run across the entire software stack.

Graphical User Interface (GUI) testing

These tests check the GUI — aka the screen the user sees. It includes menus and boxes, windows, toolbars, buttons, and icons. It’s usually a part of functional testing. When running GUI tests, the general order runs as follows:

• Run end-to-end tests on the main user journeys, including login and checkout.
• Test the functionality and security of data-sensitive functions, such as signup forms and payment tools.
• Run integration tests to ensure your app or website runs smoothly with third-party apps.

Exploratory testing

During this stage, human testers check the system on an ad-hoc, freeform basis. They’ll then record their thoughts about what they think should be tested before the checks take place. This gives you a good idea of how real people will interact with your software.

Some companies offer rewards to users who can find issues or defects by doing something unexpected. One security researcher found a bug on Instagram and was rewarded a whopping $30,000 by the platform.

Continuous delivery includes all of the above tests into the pipeline, with the end goal of checked, approved code that’s ready for deployment. If the code fails the tests, it’ll be rejected. Ideally, your testing software will then send an automatic notification to both the manager and developer so that the code can be corrected quickly. Here’s a process outline for you to follow:

• Run integration tests until something doesn’t pass
• Work out what needs to be modified
• Zoom in on the issue with unit testing
• Go back to integration testing once the units pass
• Repeat until all code passes functional testing

Software testing best practices

1. Emphasize the importance of ownership

Every line of code makes the customer’s experience better (or if done incorrectly, worse). Let your dev team know that everything they do is important.

Promote a sense of ownership. Because automatic testing removes the developer from much of the testing process, they may feel detached from quality checks and lose sight of the customer’s needs. To counteract this, empower your team to produce test coverage for the individual features they create. This means they can prove the quality of their own work.

2. Use customer feedback

The nature of software testing as part of continuous delivery means customers provide feedback regularly. This information is then fed back to the team, who can incorporate it into the next release. Use feedback to hone your tests, so that the next iteration more closely matches their needs.

3. Create a testing plan

Ideally, time, budget, and resources would not be an issue, and you can test every single thing to your heart’s content. Sadly, this is rarely the case, and there will be times when you and your team will need to rush. This is where a plan comes in handy (as well as a good understanding of triple constraint management).

Then, work out what kind of tests will be most applicable to your software. If you’re creating something that has a very interactive GUI, then you’ll need to run plenty of functional tests. Software that doesn’t have a user interface will benefit more from unit tests.

Final thoughts

Software testing is a key part of ensuring the quality of your finished product. It’s expensive but worthwhile: Delivering something without bugs quickly and efficiently shows the customer your team is efficient and professional. It’s also faster and more efficient than traditional QA testing processes, which often rely heavily on manual—and therefore, error-prone—inputs.

Starting with a well-thought-out strategy is a vital part of creating a smooth testing environment.

To really speed things up and boost collaboration, consider integrating your CD tools with project management software. This allows developers to track their own work, pull tasks through, and leave comments for the rest of the team to see. It’ll also send out automatic notifications, so everyone is alerted of any issues immediately, without the need for email. Developers can work smarter to pick up bugs more quickly and fine-tune software more efficiently. The end result? A happier, less stressed dev team — and most importantly, a happy customer.

Want to learn more about the full software development lifecycle? Make sure to check out our DevOps Guide.