Travis-CI node build failing randomly

Question

I have a problem with one node module on which all tests are working locally, on other boxes and on Travis-CI but sometimes, and not due to a timeout, one single test fails on Travis-CI. The test is deterministic, at least according to me, so it shouldn't fail randomly.

I've traced the problem to this part:

function record(rec_options) {

    //  Originaly the parameter was a dont_print boolean flag.
    //  To keep the existing code compatible we take that case into account.
    var typeof_rec_options = typeof(rec_options);
    var dont_print = (typeof_rec_options === 'boolean' && rec_options)
      || (typeof_rec_options === 'object' && rec_options.dont_print);
    var output_objects = typeof_rec_options === 'object' && rec_options.output_objects;

...

        var out = !output_objects ?
          generateRequestAndResponse(body, options, res, datas) :
          generateRequestAndResponseObject(body, options, res, datas);

For the value of rec_options being { dont_print: true, output_objects: true }, var output_objects is evaluated (again only sometimes) as false so the function outputs strings and not objects upon which the test, correctly, fails. The proof that the flag evaluation fails can be seen in this build which dumps out string instead of objects (also I have a test to ensure that the returned value is an object whenever output_objects is specified and that fails as well showing string where object was expected).

According to me this should never happen and output_objects is not defined ambiguously and certainly not randomly. And yet this is exactly what happens time after time.

What am I doing wrong?

Answer 1

The problem was in one of the unit tests that wasn't waiting for its async call to finish before finishing itself. This would then interfere, depending on the timing, with execution of other tests. This is very specific to the node module in question (nock) as it mocks HTTP/HTTPS requests and thus subsequent tests would sometimes capture the async requests from the faulty test.