How Unit Testing Improves Code Quality in TDD

Shares

How Unit Testing Improves Code Quality in TDD

Hardik Shah
in Quality Assurance
- 26 minutes
unit testing

“Tests are stories we tell the next generation of programmers on a project.” — Roy Osherove

And unit tests are the stories which maintain the software quality from the first line of code. Unit tests weed out defects at an early stage, promotes safe refactoring, comprehensive documentation, improved coupling and fewer regression tests.

In software engineering unit tests are never supposed to “replace” any other form of testing, but to be used together the whole time. Unit testing is a continuous learning process. As it is performed by developers, it is important for them to learn how to unit test, what to unit test and the best practices.

What is Unit Testing?

Unit testing is the smallest part of the code that can be tested in isolation. For object-oriented programming, you need to consider a whole class or an interface as a unit but for procedural or functional programming it could also be a single method or function. In the end, the purpose of unit testing is to validate that each unit/component of the software performs as expected.

Despite the variations, there are some common elements like Unit tests are low-level and performed by developers. Also, developers use unit testing frameworks according to the programming language, e.g. JUnit for Java and NUnit for .NET.

Let’s understand unit testing by taking a simple example of a system which performs orders and payments. Sometimes the system crashes and we want to show the message to the customers that they can’t connect to the bank. Also, we want to write unit tests so that the problem won’t occur again.

Following images shows the step by step process of our scenario which we want to test.

unit testing

Step1: It shows our system of Order-function. It calls “createOrder” when we place an order.

Step2: As a result of calling createOrder, the Order-function will, inside itself, instantiate a Pay-function.

Step 3: Now, Order-function calls “pay” on the Pay-function instance.

Step 4: Now, payment instance sends the payment request to the Bank.

Step 5: When bank’s server is down it shows an error. Here we want to write a unit test that simulates the bank returning an error. One good way to solve this would be creating a mock Pay-function (fake Pay-function), which returns a mock error.

Here we can’t write unit tests because Pay-function is integrated with Order-function and to perform unit testing we need to isolate Pay-function from Order-function.

This type of code is called as braided or coupled code. The system as a whole was much less predictable and much more prone to change than the individual components. We cannot write unit tests for Order-function, because it’s not a unit! They are two units coupled together.

How can we separate these units? See the following steps.

Step 6: Well, instead of instantiating Pay-function inside of Order-function, we instantiate it outside and inject Pay-function into Order-function.

Step 7: Here we want to test Order-function in isolation but to get the desired output we still require the Pay-function. To accomplish this, we can use Dependency Injection method. It will inject Pay-function into Order-function by keeping itself outside of Order-function.

Step 8: The Order-function is no longer responsible for creating a Payment. Instead, it now expects to be passed a Pay-function that has created outside. The real code will use the real, injected Pay-function, while the unit tests instead injects a mock Pay-function. Now, when writing our unit test, we can simply tell the mock Pay-function to throw a mock error when pay is called on it.

Now, we can test both scenarios in isolation because when we inject Pay-function into Order-function it does not create Pay-function inside Order-function.

After Dependency injection, we can easily manage dependencies between objects. Once we break coherent functionality off into manageable pieces, the code becomes more modularized and easy to do unit testing.

How to do Unit Testing?

A typical unit test is carried out in three parts. First, it initializes a small piece of code, typically a class for testing(also known as the system under test, or SUT). Second, it applies some stimulus to the system under test (usually by calling a method on it). Third, it observes the result and checks if your expectations are met. If the result is consistent with the expectations, the unit test passes. If expectations are not met, a test has been failed which indicates that there is a problem in the system under test. These three unit test phases are also known as Arrange, Act and Assert, or simply AAA.

Let’s understand it with example test scenario: When a stack is empty, pushing an item increments the count to one.

public void pushing_an_item_onto_an_empty_stack_increments_count()
{
// Arrange
var stack = new Stack();

// Act
stack.Push(false);

// Assert
Assert.That(stack.Count, Is.EqualTo(1));
}

Arrange: This is the first step of a unit testing where we need to write a code to set the specific test case. For example, it consists the instance of the objects of the classes or variables. Also, we can create mock objects and other initialization if necessary.

In the stack example, the arrange phase would simply mean creating an empty stack.

Act: This is the second step of a unit testing and it executes the test. In other words, we will do the actual unit testing and the result will be obtained from the test application. Basically, we will call the targeted function in this step using the object that we created in the previous step.

In the empty stack example, the “Act” phase occurs when Push() is called.

Assert: This is the last step of a unit testing and it checks that our expectations are met.

In the stack example, the “Assert” phase occurs when the Count property is checked against our expectation.

The benefit of using AAA method is that it clearly separates what is being tested by the setup and verification steps. Also, it clarifies and focuses on a historically successful and generally required set of test steps.

Unit testing in Test Driven Development(TDD)

Once you know how to isolate the complex code and write structures and solid tests with unit testing framework, the next step is to identify when to write the tests. In traditional development approach, unit testing comes after the code has been written.

In TDD when you start unit testing, first you have to think of an outcome you want which you don’t currently have. You code towards that outcome. You run the code to see if you now have the outcome you wanted.

Sometimes developers find it very difficult to get an idea about what they expect from writing tests first. How to get the vague idea about the functions which are not written?

When you start writing code you think about the goal and how the function should work. To achieve the goal you require appropriate input data and expect what kind of results it will give back.

In TDD, We can start this process by doing exactly that – we just won’t write any code for it yet.

Let’s understand the process unit testing in TDD by taking an example of a Table reservation system at the restaurant. We want to do unit testing for the function cancellation of the reserved table.

Now, cancellation can be done by two types of user: 1) if a user is an admin 2) user who has made the reservation. Based on the user types we need to test three scenarios:

  • Cancellation request will be accepted if a user is an admin
  • Cancellation request will be accepted if a user who made the reservation wants to cancel
  • Cancellation request will not be accepted for all other users

Step 1: Write the test case

[TestMethod]
public void CanBecancelledBy_UserIsAdmin_ReturnsTrue()
{
     
     var reservation= new Reservation();

     var result = reservation.CanBecancelledBy(new user { IsAdmin = true });

          Assert.IsTrue (result);
}

[TestMethod]
public void CanBecancelledBy_SameUserCancellingTheReservation_ReturnsTrue()
{
     
     var user= new User();
     var reservation = new Reservation { MadeBy = true };

     var result = reservation.CanBeCancelledBy(user);
     
     Assert.IsTrue (result);
}

[TestMethod]
public void CanBecancelledBy_AnotherUserCancellingReservation_ReturnsFalse()
{
     
     var reservation = new Reservation { MadeBy = new user() };

     var result = reservation.CanBeCancelledBy(new user());
     
     Assert.IsFalse (result);
}

Step 2: Write a minimal production code

At this stage, developers know should be the output based on input data. In our example, we know our output that cancellation request should pass for for two types of users and fail for other users.

In a non-TDD workflow, you might have started writing code for the function. Also, you might check other aspects while coding to know how the return value is affected by some X parameter.

This is where most people run into trouble with TDD. At this stage, you have lots of ideas on how to write function but don’t know where to start until you start writing it.

But the main advantage of TDD is that you just need to focus on the basic step instead of thinking about all the possibilities.

In our example, we need to think about the simplest possible code which can at least run the test cases. So we are writing minimal code by writing just “user” instead of defining the user.

Public class Reservation
{
      public User MadeBy { get; set; }
      public bool CanBeCancelledBy(User user)
      {
           If (user)
               return true;
      }
}

Once we run the test case it will fail because we’ve not specified the types of user whose cancellation request should be accepted.

Step 3: Refactor

Now, to get the expected output we need to refactor our code where we will specify the user type.

Public class Reservation
{
      public User MadeBy { get; set; }
      public bool CanBeCancelledBy(User user)
      {
           return (user.IsAdmin || MadeBy = user)
              
      }
}

After the refactoring, our test cases will pass because we are getting the expected output.

This was the simple example but in a complex code you just need to repeat this process from step 2. Here you need to write minimal code until your test case passes.

Technically, one of the main advantages of TDD is that you’re basically testing the test itself because you don’t need to change test when it fails. If you expect your test to fail and when it passes, there’s a possibility that your tests contain bugs or you’re testing the wrong thing. If a test failed and now you expect it to pass, and it still fails, you can make the similar assumptions like the previous scenario.

Also, TDD helps you to reduce the amount of useless code you might write because every line of code you add is verified by a test.

Unit testing best practices

Test Classes should be placed in an appropriate directory

Test code and source code need to be separated so that both can be individually built and distributed in different manners. It makes the development process more clear. The most common way is to have the same package structure in a different directory from the source code.

The test classes should be placed in the mirrored package structure under the test source.  This allows the test class to access test source easily. Any number of test base directories can also be created depending on the types of tests that are being written.

Define a standard naming convention for Test Classes

It is very important to identify between classes that represent the code that organises the tests, the code that is testing it and the code being tested. A default naming convention that may be adopted includes the following:

  • Keep the source code class name same as its original name.
  • Name of the testing class should be same as the original class but appended with the test.
  • The class that organises tests should have TestSuite appended to it.

The most popular way is Roy Osherove’s naming strategy, it’s the following:

[UnitOfWork__StateUnderTest__ExpectedBehavior]

It has every information needed on the method name and in a structured manner. The unit of work can be as small as a single method, a class or as large as multiple classes. It should represent all the things that are to be tested in this test case and are under control.

Isolate the code to test only one unit at a time

Make sure to isolate the code into small groups of classes/units to test them independently. If they are coupled together, you will have lots of overlaps between tests and changes to one unit can affect the other unit and cause failure.

Generally, in a project, all the tests are run at one time. The order in which the tests are run cannot be determined; for this reason, the tests should be independent of each other.

Avoid multiple assertions in a single test

Unit tests check how the piece of code should behave, not the whole code. If you add multiple concerns, you will not get the benefit of unit testing. To understand the problem of multiple concerns, look at the following example.

[Test]
  Public void checkVariuosSumResultsIgnoringHigherThan1001()
  {
        Assert.AreEqual (3, Sum(1001,1,2));
        Assert.AreEqual (3, Sum(1001,1,2));
        Assert.AreEqual (3, Sum(1001,1,2));
}

There is more than one test in this test method. You might say that three different subfeatures

are being tested here. Here developer has written three tests as three simple asserts to save time.

What’s the problem here?

When asserts fail, they throw exceptions. Some special exceptions caught by unit test runner which understands this exception as a signal that the current test method has failed.

Once an assert clause throws an exception, no other line executes in the test method. This means that if the first assert in above listing failed, the other two assert clauses are never executed. So? Maybe if one fails you don’t care about the others? Sometimes. In this case, each assert is testing a separate feature or end result of the application, and you do care what happens to them if one fails.

Mock out all service calls and database operations

Many times API calls to database operations and service calls overlap multiple tests. Different unit tests can influence each other’s outcome. Make sure that database or network connection is not active when you start to run your tests.

Avoid manual intervention

For effective unit testing, it is important to avoid manual interventions by developers. Developers write debug statements to track down a particular problem. The benefit of the good unit test suite is that it narrows down the area where debugs statements are needed when tests fail.

Developers use standard debugging statements like System.err.println and System.out.println which should be removed. These type of statements can be rewritten into assertion much easier. Below is the example which shows the removal of manual intervention.

Public void testMethodManually()
{
  final String ORIGINAL STRING = “MyString”;
  String convertedString = MyCode.formatStringUpper (ORIGINAL STRING);
  System.out.println (“Converted string is : ”  + convertedString);
}

This can be written as follows.

public void testMethodAutomatically()
{
  final String ORIGINAL STRING = “MyString”;
  final String EXPECTED STRING = “MySTRING”;

  String convertedString = MyCode.formatStringUpper (ORIGINAL STRING);
  assertEquals (“Converted string was not expected, EXPECTED STRING, convertedString );
}

Another advantage of this technique is that it demonstrates the intention of the test to other people who may read the code.

Benefits of Unit Testing

Find defects at the early stage of developing

This is the main advantage of unit testing. As unit testing is the first testing effort performed on the code, it is easier to find the bugs at the beginning of the development. Early bug-detection is cost-effective for a project. The code fixes becoming more expensive the later they’re found in the life-cycle.

Bugs tend to multiply, so if they don’t get fixed at an early stage, there will be more bugs you need to fix. When there are multiple bugs, the process can take a lot longer because you must sort through other errors.

Unit tests enable safe refactoring

Refactoring is a common activity while coding a complex application. When developers build more functionalities code become more complex and hence code changes occur frequently. A complete set of unit tests will ensure that a refactoring does not introduce an unexpected bug.

In TDD, refactoring done on the minimal code that has been written to pass the test case. So here you need to refactor the minimal/basic code instead of complex code. As unit tests are written before the code it reduces the amount of useless code you might write. The reason is that every line of code you add is verified by a test.

Consider you are going for a major product change which requires changing the database from MySQL to MongoDB. With unit tests in place, you can easily estimate the time required to migrate the database and execute refactoring of the product.

Provides Documentation

Unit testing also gives you living documentation about how small pieces of the system work.

One interpretation is that unit tests are “executable documentation”. You can run the unit tests against the code and it will tell you whether it’s still performing as it was when the tests were written to pass, or not. In that way the unit tests “document” the functionality of the system at some point in time, in an executable way.

Less regression testing

Regression testing is the process of identifying unexpected defects in previous scenarios when there is a bug fix or new functionality is added to the existing ones. Defects which are found after regression testing take more time and efforts to get fixed.

Your unit test suite guards assiduously against these regression defects. And, it provides concrete evidence that new functionality behaves as expected. All of this adds up to the very real business outcome of deploying with more confidence. You’re far less likely to need to dedicate time to support calls, triage, and patching. And you’ll look better to your users.

Generally, regression tests are a combination of unit test cases and integration test cases. Also, it is better to create a logical batch of such tests cases in the form of comprehensive test suite instead of having one large regression test.

“This simply means that more the unit tests are effective, the less regression tests you need to write.”

The development process becomes more flexible

Sometimes it is important to fix the bugs at an early stage. Despite the best efforts, some bugs remain in the program and feature may stop working. Now, you have to fix the problem immediately. It is called the hotfix. When you run unit tests with the fixes applied they should reveal undesirable side-effects. Unit tests reduce the requirement of hotfixes because defects can be found and fixed at the time of developing. Even if you required to do a hotfix, a unit test suite will help you to make it easier without introducing new problems.

Improves Coupling in the Software Architecture

If you write the tests upfront as you should in unit testing, you have two alternatives. Either you create a hugely complex code with lots of mock objects, or your code in a way that makes it easy to call in isolation. Thus the practice of testing units inherently lowers the coupling in the architectural pattern. Change in one service (unit) does not affect the other units. Code reuse is easier as there is no dependence of other units.

On the other hand, if your application is coded with a high degree of coupling. Well, my condolences!! You will experience a lot of pain as you try to squeeze the code into your test environment.

Limitations of unit testing in Traditional approach

Despite having all the benefits of unit testing, there are some limitations you should consider. Unit tests are not a silver bullet that provides a bug-free software.

It does not test code dependency

Unit tests do not test the interaction between two units or classes and hence you can’t test how one unit behaves with another. For example, there are two components where component A sends a request to call the data and another component B receives that request and return that data. Here unit tests don’t check this interaction. Both the cases can be tested individually. This scenario requires integration testing.

Multiple bugs in production code

When bugs can be found in production code instead of unit test suite, you will find that bug is available at two places: the original bug in the application code and another one in the test code. For example, in a web application, there is a feature that allows a user to reset the password without logging in. Here unit test framework performs an automatic login which makes sense for 99% of the tests. But if you add an additional code as a side effect where a user requires a login then unit test hid that bug and allows automatic login. The conclusion of this example is that when bug can be found in production code instead of unit test code, the chances for the two bugs occurring together are higher than for the first bug occurring alone.

Time Consuming

Writing unit tests requires a lot of time and efforts. If you want to cover entire system with unit tests having high code coverage, you are talking about a hell lot of time to be spent. Because everytime you encounter a bug in the production code you need to go back to unit tests and add a specific test case to ensure that the bug should not occurs again.

Developers should avoid writing unit tests for trivial code because it consumes lot of time. Instead, it is more cost-effective to write a test that checks a piece of functionality that would fail if the property did not work properly.

Not feasible for database and network layer

Lots of your code will talk to the network, query a database, draw to the screen, capture user input, and so on. The code responsible for all of that is impure, and as such, it’s a lot harder to test with unit tests.

You always mock the repository when dealing with code that passes data to the repository, and you mock the ORM’s hooks when developing within the repository itself. While there are “database unit tests” and you can “mock” a database using something like SQLite, you’ll notice that these tests just never fit at the unit level.

Case Studies of Unit testing in TDD

As we’ve seen, there are lots of benefits of doing unit testing in TDD but it’s important to know the experience of the individuals/companies who perform unit testing in TDD. For that, we’ve asked some questions to the people who’ve been working in TDD.

Case study 1:

unit testing

What’s the difference you’ve noticed in defect rate in unit testing when you started working in TDD?

When I started practising TDD, my personal defect rate decreased by over 90%. Clearly I had a very high defect rate at the time. More importantly, though, I noticed that when I practised test-first programming (even without the emphasis on refactoring), I produced different kinds of defects from what I had done before. I would still produce the occasional very silly mistake and I would still produce very complicated, difficult-to-diagnose defects from unexpected behavior emerging from integrating larger parts of the system, but 99% of the non-trivial-but-easy-to-understand defects disappeared, because I find those mistakes myself.

I now think more in terms of the cost of mistakes instead of the frequency of mistakes. Practising TDD helps me reduce the cost of mistakes by reducing the time between when I make the mistake and when I notice the mistake. Herein lies a lot of the cost-saving of test-first programming: the sooner I see the mistake, the more easily I fix it.

According to Bhat and Nagappan’s case study at Microsoft showed that development time using TDD grew by 15-35%. What is your experience with development time?

I don’t know the case study well, so I don’t know whether they compare apples to apples. When I worked at IBM, I worked in the common enterprise environment that had “code freezes”. Before code freeze, programmers built new features, and after code freeze, programmers only fixed the most urgent manager-approved defects. After I started practising TDD, I delivered probably 20% less value up to code freeze, but after code freeze, I earned the trust of the managers to deliver more features right up until the date of release. I can’t quantify the additional value of that trust, but I found it more intuitively valuable over the long term than squeezing out 10% more code over the short term.

Moreover, given the evolutionary design aspect of TDD, I noticed greater longer-term benefits, since I could build version 4 on top of the same code from version 1. I didn’t need to throw everything away every 2-3 years. This savings certainly compensates for going even 20% slower over the short term.

Both Defect rate and Development time impacts on Cost of development. Have you noticed cost reduction after implementing TDD at your organization?

I mostly noticed increased cost certainty, meaning a large decrease in unexpected/unplanned costs. Although I didn’t stay at IBM long enough to witness it myself, I know that when cost uncertainty decreases, then cost itself also gradually decreases.

In most cases, TDD results in good test coverage and a lasting regression test suite. Did you experience any increase in test coverage? If yes, then how much?

The difference was night and day. When I practice TDD, I get the 85% coverage that matters most as a natural part of the process of writing code.

Case study 2:

What’s the difference you’ve noticed in defect rate in unit testing when you started working in TDD?

I can relate a couple stories. I worked with an insurance company who deployed a moderately-sized (~100,000 SLOC) test-driven Java app to production. In its first 12 months, they uncovered only 15 production defects total. This is dramatically less than a typical production application.

I worked as a Clojure programmer from 2013-2016 using TDD. I did not code any “logic” defects during this time. We did have integration-related defects and defects related to the misunderstanding of customer interests.

In the work I do with customers, the code that we test-drive does not exhibit logic defects for intended behavior.

TDD does not remove all defects. Among other classes of defects, you will still have integration-related defects, defects related to the misunderstanding of requirements or missed requirements, and defects for unexpected interactions, but even these begin to reduce in number when TDD is employed.

According to Bhat and Nagappan’s case study at Microsoft showed that development time using TDD grew by 15-35%. What is your experience with development time?

I believe the studies indicate that TDD creates an increase in “initial development time” in comparison to other projects. These figures, as far as I know, do not include the costs of rework due to defects, or other costs associated with defects; nor do they consider the increased cost in long-term development of code when the quality of code (in absence of continual refactoring) decreases.

Here you can see studies of Test-driven development which provide some good summaries.

I take little stock in any one study. That most of the studies show similar results, however, suggests that the costs/values attributed to TDD are reasonably in line with reality.

From a personal stance, I firmly believe TDD has allowed me to increase my development speed over time as the codebase for any given project grows in size. I can and have related anecdotes about how not practicing TDD on efforts increased even after a very short amount of time.

During my Clojure development years, we created a significant amount of code in a short time using TDD. I do not believe we would have gone any faster by abandoning TDD, particularly given the dynamic nature of changing requirements.

Both Defect rate and Development time impacts on Cost of development. Have you noticed cost reduction after implementing TDD at your organization?

Things like this are always hard to quantify. That our Clojure codebase had few logic defects indeed meant we had reduced costs in the areas of rework, defect management, and support. I also believe that we had reduced costs in terms of time required to understand current code behavior.

In most cases, TDD results in good test coverage and a lasting regression test suite. Did you experience any increase in test coverage? If yes, then how much?

Since we were doing TDD on most of the Clojure code, our coverage percent, at least on the system I worked on, was likely in the high 90% range by definition. We never measured it, as it was not a relevant number (if everyone is practicing TDD correctly, there is no need to track the coverage). In the past, I’ve test-driven systems and later come back to measure code coverage out of curiosity; in these cases, the numbers have always been in the high 90% range. Certain areas of code–pure view code with no real logic–end up having low or no coverage, but that’s the point of removing all logic from it.

Case Study 3: 


What’s the difference you’ve noticed in defect rate in unit testing when you started working in TDD?

Before using TDD there were a lot of things that were caught later on by QA and thrown back to developers. I started at Babbel right away doing TDD, so here there’s no improvement measured. In my old company, after we adopted TDD the issues reported by QA decreased considerably. We used to have on average 5 issues popping up on QA and this went down to 1 or 2. We also used to have a 95% free-crash rate and it increased to 98%.

According to Bhat and Nagappan’s case study at Microsoft showed that development time using TDD grew by 15-35%. What is your experience with development time?

The development time increased for sure, but what we lose in developing time we gain in less QA and fewer bugs in the long run.

Both Defect rate and Development time impacts on Cost of development. Have you noticed cost reduction after implementing TDD at your organization?

Hard to say. We develop for ourselves, meaning we don’t really build apps for other clients where we would charge per hour. As explained, you definitely notice an increase in developing time and a decrease in the number of issues reported by QA. In general, we spend less time releasing new features and therefore I guess one could say it’s cheaper to develop.

In most cases, TDD results in good test coverage and a lasting regression test suite. Did you experience any increase in test coverage? If yes, then how much?

Yes definitely. This is actually something we actively measure here at Babbel. It increased about 40%.

Conclusion

Of course, you can release your software faster without unit testing but later users may find bugs. The recipe of building better software is simple: unit test early with the build, do it regularly and refactor when required.

Unit testing is a great skill for any software engineer to develop as it can help in writing less bug-free code. If you are testing with TDD approach, make sure you write quality unit tests. In this blog, we’ve discussed how to do unit testing in TDD and best practices which will help you to improve your unit testing skills.

Hardik Shah

Working from last 8 years into consumer and enterprise mobility, Hardik leads large scale mobility programs covering platforms, solutions, governance, standardization and best practices.

test coverage

Stomp out bugs with Unit Testing

Like what you are reading? Sign up now to get the further updates on unit testing and TDD. 

You have Successfully Subscribed!