How to Test Custom Lint Checks

Sven Vidak — Thu, 27 Jun 2019 09:30:00 +0000

In my previous article, I’ve talked about writing Lint checks and benefits they can bring to your project. I’ve also outlined how to write custom Lint checks, but I didn’t cover how to test them.

Of course, you can always observe if Lint reports an error or warning if you make a mistake, but the problem arises if you have a bug in your implementation.

Tests help make sure the check behaves correctly in every possible case, but can also help with debugging.

Dependencies

First step is to add test dependencies in your module’s build.gradle file:

testImplementation "com.android.tools.lint:lint:$lintVersion"
testImplementation "com.android.tools.lint:lint-tests:$lintVersion"

lintVersion is defined in the same way as in the previous article – gradlePluginVersion + 23.0.0. At the moment of writing, it would be 26.4.0.

Basic test class

Lint checks are tested with the JUnit framework. The only part that’s a bit different is that your class must extend LintDetectorTest class which will force you to specify detector and issues you want to test. In the previous article, I’ve defined all components that one Lint check has to have. I’ll use those here.

When you extend LintDetectorTest class, you’ll have to implement two methods: getDetector and getIssues.

class CorrectClickListenerDetectorTest : LintDetectorTest() {
    override fun getDetector() = CorrectClickListenerDetector()
    override fun getIssues() = listOf(ISSUE_CLICK_LISTENER)
}

For each test, you’ll need a code sample that Lint will analyze. You can define those in any way you want (as a variable, read from file, etc.), but have in mind there are differences when it comes to Java and Kotlin.

Java code sample must be syntactically correct. For example, you can call non-existing methods on any object and you don’t have to specify imports for used classes, but parenthesis, keywords, etc. must be correct.

private val incorrectMethodCallJava = """
    public class Test {
        private static String s1 = "Test string 1";
        private static String s2 = "Test string 2";

        public static void main(String[] args) {
            s1.setOnClickListener();
            s2.trim();
        }
     }
    """.trimIndent()

As you can see, there are no imports and we call setOnClickListener on a String object which does not make any sense, but Lint is cool with that.

On the other hand, when defining a Kotlin sample code, you have to make sure that the code will compile if you copy it into a real project. Since we are testing method calls on Android View, you have to specify import statement for the View class and somehow obtain instance of that class so you can call a method on it. Luckily, we don’t have to drag beloved Context to our test – it’s enough to have a method that takes a View instance as a parameter.

private val incorrectMethodCallKotlin = """
    import android.view.View
    class Test {
        fun main(view: View) {
            view.setOnClickListener { }
        }
    }
  """.trimIndent()

Have in mind that having import statement in the code sample does not require you to include Android SDK as a testing dependency – all dependencies you need are the two mentioned at the beginning of the article.

First test

Each test is defined as a method annotated with @Test (same as any other JUnit test), It’s written as a chain of commands that tells Lint what to analyze, and what assertions to test.

@Test
fun incorrectMethodUsage_shouldReportAWarning() {
    // obtain Lint analyzer
    lint()
        // give it a list of files to analyze
        .files(java(incorrectMethodCallJava))
        // run analyzer
        .run()
        // make assertions
        .expectWarningCount(1)
}

Kotlin file(s) can be analyzed in the same way, you just have to pass kotlin(incorrectMethodCallKotlin) as a parameter to files method.

More assertions

In the previous article, I’ve defined a quick fix option for a check. Lint testing API allows you to test if the fix works.

private val correctMethodCallJava = """
    public class Test {
      private static String s1 = "Test string 1";
      private static String s2 = "Test string 2";

      public static void main(String[] args) {
          s1.setThrottlingClickListener();
          s2.trim();
      }
    }
""".trimIndent()

@Test
fun fix_shouldUseCorrectClickListener() {
    lint()
        .files(java(incorrectMethodCallJava))
        .run()
        // check that the fix produces the same code as the one given as a second parameter
        .checkFix(null, java(correctMethodCallJava))
}

Also, you can match exact output of your check.

private val incorrectMethodCallWarning = """
    src/Test.java:6: Warning: Use setThrottlingClickListener [UnsafeClickListener]
            s1.setOnClickListener();
            ~~~~~~~~~~~~~~~~~~~~~~~
    0 errors, 1 warnings
""".trimIndent()

@Test
fun incorrectMethodUsage_shouldShowCorrectWarningMessage() {
    lint()
        .files(java(incorrectMethodCallJava))
        .run()
        /* this check is sensitive to indentation so you might have to adjust warning variable a bit due to article formatting */
        .expect(incorrectMethodCallWarning)
}

And, of course, you want to check that your check won’t report any issues.

@Test
fun correctMethodUsage_shouldNotShowAnyWarningMessages() {
    lint()
        .files(java(correctMethodCallJava))
        .run()
        .expectClean()
}

A good starting point

There are many more assertions that enable you to verify the correctness of your check (you can even check whole reports).

The checks in this article can cover a lot of cases which are a good starting point for every type of a check that exists. For those eager to learn more, here you can find tests for XML checks. That’s all folks. Happy testing.