Added

  • Support CSV and JSON Lines formats for exporting results
  • Emit requeststarted, requestskipped, requestfinished, requestfailed, maxdepthreached, maxrequestreached and disconnected events.
  • Improve debug logs by tracing public APIs and events.

Changed

  • Allow onSuccess and evaluatePage options as null.
  • Change crawler.isPaused, crawler.queueSize, crawler.pendingQueueSize and crawler.requestedCount from read-only properties to methods.

Fixed

  • Fix a bug of ignoring maxDepth option.