- Support
obeyRobotsTxt
option - Support
skipDuplicates
option - Make
cache
option to be required - Added docs for
exporter
option - Modify
BaseCache
interface
Fixes: https://github.com/yujiosaka/headless-chrome-crawler/issues/11
obeyRobotsTxt
optionskipDuplicates
optioncache
option to be requiredexporter
optionBaseCache
interfaceFixes: https://github.com/yujiosaka/headless-chrome-crawler/issues/11