If you want to write unit tests on JS classes, sensibly, you need a test framework. Unit testing quite alot of client-side JS code is hard, as it is quite involved with the DOM or webrowser. I present a range of options. Please remember 'does it feel intuitive' isn't something that can be put in a unit test. I am not presenting my own code here, as the current range of tools seem adequate (I have made a few tools previously, but those projects are firmly one of several companies IP. The publically available tools should be better looked after and have more investment, so represent a better solution.).
I repeat to be clear, this article is about unit testing JS modules, not behaviour testing webpages. The I am paid for is generally complex enough that one needs to test each module, to reduce later stage costs. I will write an article on BDD tools later.
Metrics I am using for comparing test frameworks:
- what you can test with X;
- complexity / time to make stable tests;
- ability to integrate with a common CI tool;
- ability to output TAP 1 (for linking into other tools);
- fragility of tests;
- are tests inside browser, or outside
Writing easy-to-test JS is more complex than other languages. Nicely written Objects ~ JS style ~ do not export many symbols. Most in-browser code is written a pluginable libraries; and each library must allow that there are other random libraries in use at the same time. All the in-browser code shares a single namespace per page. It is possible to achieve fake Namespaces, by creating your object as a child object e.g. com.myCorporation.UsefulLibrary won't clash with UsefulLibrary or com.MonopolyCorpPLC.UsefulLibrary. Pls see 2 or others for more detail. Technically this isn't a namespace (compared to other languages). Another approach is to use SEAF / IIFE ~ there are many abbreviations to this effect ~ more detail at 3 or 4. After the widespread adoption of jQuery 5, the second approach is really common.
Writing easy-to-test JS is more complex than other languages. If you don't have access to an objects methods, how can you unit test? How can you see side effects? How can you see Exceptions? Please consider this when designing your architecture . . .
- Selenium ~ no demo code practical on this site ~ this is a simulation of a full webrowser. Selenium 2 is written in Java.
- strengths: Massive numbers of features, just like webrowsers. Comprehensive ability to automate any possible action in a webrowser. Can be used by non-tech people, when used in record mode. Can easily test in each browser (although they would all need to be installed locally).
- weaknesses: Tests cant be quite slow, when simulating mouse movements. Need to manage a JDK. Unless carefully written, scripts will go out-of-date, compared to the target site (i.e. fail to run for nonsense reasons). It is hard for the Selenium team to support newest CSS, although this wouldn't break JS tests. This is designed as a site tester not a JS module tester.
- Jasmine is a unit-tester, similar to any other xUnit tool. It is available from git with docs. It runs in-browser, or a pretend browser such as gulp; and has bindings for Node, Python and Ruby projects. TODO: add demo.
- strengths: nice shiny xUnit interface. Can be run in a CI platform 6 7 8. It is designed to be easy to read 9 , and borrows ideas from rspec, and related tools. As part of BDD rather than TDD, it defines the setup()/teardown() pair as beforeEach() and afterEach(); but ignoring the words this has the same effect. Jasmine can be extended to support TAP with 10
- weaknesses: I think the Jasmine scripts are quite long, apparently so do other developers as there are several script generators eg. 11. Early versions of Jasmine has poor support for async events (which are really common in JS). However this is fixed by now 12.
- Qunit ~ demo code 13 ~ A small and light unit tester, for JS only (i.e. it can't test CSS or HTML). I use this for my published jquery modules.
- strengths: Simple and concise API. As it is just 2 files, it is light in the project repo. If you are demoing that a JS unit is correct, without concern for a large test suite, it is very nimble. Good for tests that need to be run a part of the DOM (i.e. you want to see that a particular DOM element is set to a value). Can be combined with swarm project to run on all webrowsers at once. Can be made to output TAP, via qunit-tap.
- weaknesses: This needs to be run in a webrowser. It will output DOM manipulations, as test results. As such it is less use for integration with a CI/CD tool, like Jenkins. No test ability to test CSS or HTML.
- Behat/mink/sahi stack ~ this is same idea as Selenuim. I associate it with Symfony, and it Behat is written in PHP. Behat implements the same Domain language as Cucumber. The stack can wrap Selenuim2 as a second tech implementation. Everzet has published some introductory docs for mink 14. These state that one may run JS from inside a PHP test. The examples that are supplied aren't very useful, but would be easily reproducible. As this is also a full browser, you can do more complex testing. Mink is an abstraction layer, to allow more testing tools to be used; Sahi 15 is a Java & JS page automator, like Selenuim1. Sahi acts as an active HTTP proxy.
- strengths: With respect to testing JS, unit tests can test everything that appears in a browser. Like Selenium, test conditions can be applied against HTML, CSS 16 and JS state 17.
- weaknesses: If you are one of those people that dislike PHP, its PHP. The API for JS is less clean, as your basic test is in PHP. As the test is not native to JS, the event or async stuff is messy. I can't find a integration between Behat (the test schedular tool) and any TAP. This is lots of different bits of technology, so tests are more likely to break than a simpler architecture (four languages when using Sahi, PHP for Behat & Mink, Java and JS for Sahi, and C++/C and JS for the browser).
- Mocha/phantomjs stack, has a full headless-browser browser implemented in JS git. Mocha is another BDD spec tool 18, written in JS. Moncha is pretty much the same as every other spec tool, see 19, as example. There is phantomjs documentation. As a webrowser, phantomjs isn't a test tool; there are extensions git git 20 . Phantomjs does have evaluate(), which is functionally eval(). Twitter use Phantomjs with Qunit 21 as a another decision making API.
- Strengths: Mocha supports TAP 22. Phantomjs has close match to Chrome, Safari and a few minor browsers, as it links webkit. As a node project, Mocha links to common CI tools eg 23. For a pure JS unit (no DOM, not much DI), it would be faster to omit phantom i.e. 24
- Weakness: Unfortunately, this is old webkit. Some people (in particular, the Protractor team 25) discourage use of Phantomjs.
- Facebook inc use jest as a test tool. I think this is mostly BDD, but I didn't get the API to confirm.
- UPDATE: There is a test framework trying to replace Selenium; but written in JS, and only supporting JS. Cypress git claims to be designed for UI testing, integration tests, and unit-tests. Some links 26 27 this is an advert 28. Cypress docs claim it integrates with CI 29, but as a JS framework, they have never heard of TAP. The advert link talks about using jQuery selectors, and having a design objective of stable tests. The massive problem with this is currently it only supports chrome environment; which isn't the browser that normally has issues.