Added
- Support
waitFor
for crawler.queue()'s options. - Support
slowMo
for HCCrawler.connect()'s options.
Fixed
- Fix a bug of not allowed to set
timeout
option per request. - Fix a bug of crawling twice if one url has a trailing slash on the root folder and the other does not.