Continuous Integration Best practices: #2 Always run the tests
Staff Blogs June 4, 2013 By Manuel de la Peña Staff
Following with these blog posts series about good practices in Continuous Integration, I want to talk about the benefits of running tests.
Practice 2: Always run the tests
- Locally: a developer can use some ant targets to run test in his/her own workspace, so he/she can test the code before sending it. Please read the wiki page explaining the Testing Infrastructure in related assets:
- ant test-unit: execute all unit tests (dependencies to other systems, i.e. databases, are not real: we mock what we need)
- ant test-integration: execute all integration tests (dependencies to other systems are real, not mocked)
- ant test-class: to execute only one test class
- After sending a pull request: we use Jenkins as CI server to manage all our CI processes, and we have made that every pull request sent to a peer reviewer is monitored by the CI server: it checks out the code, and execute some tasks (compilation, format source, test execution...) The cool thing here is that there is a Jenkins plugin that can monitor the pull request and operate after tests results, managing Github pull request (auto closing if it breaks tests, writting comments, changing pull status...). In this scenario, a peer reviewer knows if the pull request he/she is going to review is good or breaks something, so we reduce the feedback loop a lot with this process, discarding bad pulls as soon as possible.
- Jenkins logs: you can configure Jenkins to send the commiters an email with test results, telling them that their commits produced a breakage. In this case we have improved the usability of default Jenkins email, to make its reading easier.
- Github logs: the plugin we use to monitor pull requests can write comments on Github, so this really good platform also sends emails when a pull is commented with tests results. So a developer inmediately knows whether his/her commits passed the tests or not.
Both of them produce very good complementary information a developer will know what to do with. Then, try to notify them with every break your CI system discovers, so the culprit can be ready to solve it as soon as possible, as we saw in last practice.
That's all for today, please wait till next blog entry about CI best practices!