This allows to specify the behavior on recursive operations, such that one can collect failures instead of dying on the first failure.
We now create the necessary directories and files for the tests on-the-fly.
