java.lang.NullPointerException: at org.archive.processors.util.RobotsExclusionPolicy.disallows(RobotsExclusionPolicy.java:156) at org.archive.crawler.prefetch.PreconditionEnforcer.considerRobotsPreconditions(PreconditionEnforcer.java:195) at org.archive.crawler.prefetch.PreconditionEnforcer.innerProcessResult(PreconditionEnforcer.java:129) at org.archive.processors.Processor.process(Processor.java:118) at org.archive.crawler.framework.ToeThread.processCrawlUri(ToeThread.java:295) at org.archive.crawler.framework.ToeThread.run(ToeThread.java:150)

Users with the same issue

You are the first who have seen this exception.

Write tip

Know the solutions? Share your knowledge to help other developers to debug faster.