Added
- Emit
newpage
event. - Support
deniedDomains
anddepthPriority
for crawler.queue()'s options.
changed
- Allow
allowedDomains
option to accept a list of regular expressions.
newpage
event.deniedDomains
and depthPriority
for crawler.queue()'s options.allowedDomains
option to accept a list of regular expressions.