Website Evidence Collector 1.0.0 Switch to the latest release

1.0.0 / 2021-01-07

• fix an issue when tool crashes when the browsing of multiple random pages with --max includes PDF files and other downloads: selected links are tested first to have mime type text/html using a HEAD HTTP request outside of the browser.
• bugfix: stop script and log error in case of unreachable websites
• feature: allow for custom page load timeouts, example: --page-timeout=200 for a timeout of 200 ms.
• feature: add option --dnt to send DO-NOT-TRACK (DNT) HTTP header
• feature: add option --dnt-js to both send DO-NOT-TRACK (DNT) HTTP header and set navigator.doNotTrack property accordingly
• bugfix: Chromium retains cookies with empty name and value if web servers send empty HTTP Cookie Header, i.e. "Set-Cookie: "
• feature: add option --task-description to include custom text/JSON in the output
• bugfix: HTML output generation crashed for some websites with incomplete SSL certificates
• documentation: include tips for fast/parallel evidence collection in FAQ
• bugfix: injected JS produces error on some websites (Uncaught TypeError: this.each is not a function at NodeList.collect (prototype.js:293))
• bugfix: tool crashes sometimes if no redirects from HTTP are found
• bugfix: catch Error: Protocol error (Page.captureScreenshot): Unable to capture screenshot
• bugfix: catch Error: Protocol error (DOMStorage.getDOMStorageItems): Frame not found for the given security origin
• bugfix: tool crashes sometimes when page response is for some reason empty (TypeError: Cannot read property 'request' of null)
• bugfix: tool hangs when page has forms that contains an element with id action
• feature: allow to pre-install cookies with --set-cookie to e.g. indicate given/rejected consent
• documentation: add to FAQ tips on data analysis with shell tool jq
• breaking change: change commandline option name of website-evidence-reporter from --output to --output-file to mark the distinct meaning and avoid clash when setting WEC_OUTPUT environment variable (short option name stays -o for both)
• update: node dependencies and filterlists
• feature: respect environment variables such as WEC_MAX=10 or WEC_headless=false
• feature: introduce option --browser-options <browser options> as an alternative to the -- <browser options> syntax (required for Docker setup)
• bugfix: update yargs dependency to get bugfix yargs/yargs-parser#261 for --browser-options starting with double dash --
• feature: add commandline option --browser-profile (-p) to allow the reuse of existing browser profiles (thanks to Hamburg DPA staff!)
• feature: include medium.com and tiktok.com links in social media subsection of output link list
• bugfix: handle non-zero testSSL error codes according to https://github.com/drwetter/testssl.sh/blob/3.1dev/doc/testssl.1.md#exit-status
• bugfix: catch and log error "Frame not found for the given security origin" (thanks to Hamburg DPA staff!)
• bugfix: fix CLI option to pass custom templates (#38)