Added

  • Emit newpage event.
  • Support deniedDomains and depthPriority for crawler.queue()'s options.

changed

  • Allow allowedDomains option to accept a list of regular expressions.