Added
- Add [HCCrawler.defaultArgs()] method.
- Add
requestretried
event.
changed
- Use
cache
option not only for remembering already requested URLs but for request queue for distributed environments. - Moved
onSuccess
,onError
andmaxDepth
options from HCCrawler.connect() and HCCrawler.launch() to crawler.queue().