We have decided to build our current product using Kotlin and, like with any new project, do everything by the book (and try to keep it that way as long as possible :D). This meant a number of automated safeguards and checks, including having good test coverage from the start. We decided to go with Jacoco as it seemed to have the best compatibility with Kotlin and set the bar as high as possible: 100% branch coverage. Yes, branch coverage. Not line coverage. It is too easy to have good line coverage and useless tests. This topic has been covered many times by others so I will not go into details. For more info you can check out e.g.: https://linearb.io/blog/what-is-branch-coverage/. The check is part of the automated build and is done on each Pull Request. No code will be merged if the coverage is not high enough. These are CICD basics but I wanted to reiterate this point. 

Of course it is still possible to write tests that have high branch coverage and tests nothing (although harder than for line coverage). It is possible to tackle this using mutation tests. Mutation tests change (mutate) the code and check if any of the tests fail. If a mutation passes all the tests, it means that the tests are of poor quality. This is slow so we run it only once a day but that is enough to keep us in check.

100% coverage sounds extreme but the assumption was to see how long we can maintain that and lower it if such need arises. If the coverage needs to be dropped we had to have a good reason for it. We have worked in this setup for over half a year now and few interesting outcomes have come from this.

Kotlin and Jacoco work well together, but not perfectly. The coverage limit had to be dropped to 98% due to some edge cases. Unfortunately as percentages work, this means that, as codebase grows, the number of branches that fall into the 1% also grows. Some critical branches might go unnoticed. We need to actively check the reports and see what actual other branches are currently not covered. Hopefully in the future we can improve on this.

No bugs reported so far were from introduction of regression to the code.

Having said that, here is the kicker. No bugs reported so far were from introduction of regression to the code. I had this realization quite recently and made me curious as to what could be the cause of that.

First is the void/null safety enforced by Kotlin. No other popular JVM language has this functionality. This is a huge boost to productivity and helps tremendously in keeping the code correct. If a variable/field can be null, the programmer is forced to handle it. This means that a visible branch is created. Jacoco can report that and force us to write a test that covers it. 

This has another effect. Since humans (that includes developers) are lazy by nature, they want to write as little code as possible. If you push an optional parameter through multiple layers and execute a number of operations on it, Jacoco will force you to write a test for each case. Some cases might not be possible to test at all.

data class User(
    val firstName: String?
)

fun someUseCase1(user: User) {
    placeOrder1(user.firstName)
    sendEmail1(user.firstName)
}

fun placeOrder1(firstName: String?) {
    if (firstName == null) {
        throw Exception("First Name is missing!")
    }
    // Do something
}

fun sendEmail1(firstName: String?) {
    if (firstName == null) {
        throw Exception("First Name is missing!")
    }
    // Do something
}

No one wants to do that. It forced us to resolve the null object as early as possible.

fun someUseCase1FewerBranches(user: User) =
    (user.firstName ?: throw Exception("First Name is missing!"))
        .let { firstName ->
            placeOrdere1FewerBranches(firstName)
            sendEmaile1FewerBranches(firstName)
        }


fun placeOrdere1FewerBranches(firstName: String) {
    // Do something
}

fun sendEmaile1FewerBranches(firstName: String) {
    // Do something
}

We end up with an easier to test, more readable and safer code. And less of it. It is also just good design that should have been done from the start. Now we have an automated tool that makes us do it.

An aspect of this is also wider usage of the null object pattern. Say we have a list of strategies to select from or an optional function parameter.

fun someUseCase2(users: List<User>?) =
    users?.forEach(::someUseCase1)

Instead of having separate tests to handle cases where the selected strategy is missing or the parameter has not been passed, we can introduce sane default behaviour (e.g. use empty list instead of optional list).

fun someUseCase2FewerBranches(users: List<User> = emptyList()) =
    users.forEach(::someUseCase1)

All those worked in tandem to guide us in having a better quality solution and allow us to keep safely introducing new features. Although there are also other reasons for it, our productivity per developer has not dropped since the beginning of the project (and it is always highest in the beginning when there is no code :D). Bugs still happen and always will. They are, however, fewer of them and of different nature. It has only been a few months on this project so I am curious how will this evolve but so far it looks very promising.