Added

  • Automatically collect and follow links found in the requested page.
  • Support maxDepth for crawler.queue()'s options.