Please report security issues only to
scrapy-security@googlegroups.com. This is a private list only open to
trusted Scrapy developers, and its archives are not public.

Well-written bug reports are very helpful, so keep in mind the following
guidelines when reporting a new bug.

check the FAQ first to see if your issue is addressed in a
well-known question

if you have a general question about scrapy usage, please ask it at
Stack Overflow
(use “scrapy” tag).

check the open issues to see if it has already been reported. If it has,
don’t dismiss the report, but check the ticket history and comments. If you
have additional useful information, please leave a comment, or consider
sending a pull request with a fix.

search the scrapy-users list and Scrapy subreddit to see if it has
been discussed there, or if you’re not sure if what you’re seeing is a bug.
You can also ask in the #scrapy IRC channel.

write complete, reproducible, specific bug reports. The smaller the test
case, the better. Remember that other developers won’t have your project to
reproduce the bug, so please include all relevant files required to reproduce
it. See for example StackOverflow’s guide on creating a
Minimal, Complete, and Verifiable example exhibiting the issue.

the most awesome way to provide a complete reproducible example is to
send a pull request which adds a failing test case to the
Scrapy testing suite (see Submitting patches).
This is helpful even if you don’t have an intention to
fix the issue yourselves.

include the output of scrapyversion-v so developers working on your bug
know exactly which version and platform it occurred on, which is often very
helpful for reproducing it, or knowing if it was already fixed.

The better written a patch is, the higher chance that it’ll get accepted and
the sooner that will be merged.

Well-written patches should:

contain the minimum amount of code required for the specific change. Small
patches are easier to review and merge. So, if you’re doing more than one
change (or bug fix), please consider submitting one patch per change. Do not
collapse multiple changes into a single patch. For big changes consider using
a patch queue.

The best way to submit a patch is to issue a pull request on GitHub,
optionally creating a new issue first.

Remember to explain what was fixed or the new functionality (what it is, why
it’s needed, etc). The more info you include, the easier will be for core
developers to understand and accept your patch.

You can also discuss the new functionality (or bug fix) before creating the
patch, but it’s always good to have a patch ready to illustrate your arguments
and show that you have put some additional thought into the subject. A good
starting point is to send a pull request on GitHub. It can be simple enough to
illustrate your idea, and leave documentation/tests for later, after the idea
has been validated and proven useful. Alternatively, you can start a
conversation in the Scrapy subreddit to discuss your idea first.

Sometimes there is an existing pull request for the problem you’d like to
solve, which is stalled for some reason. Often the pull request is in a
right direction, but changes are requested by Scrapy maintainers, and the
original pull request author haven’t had time to address them.
In this case consider picking up this pull request: open
a new pull request with all commits from the original pull request, as well as
additional changes to address the raised issues. Doing so helps a lot; it is
not considered rude as soon as the original author is acknowledged by keeping
his/her commits.

When writing GitHub pull requests, try to keep titles short but descriptive.
E.g. For bug #411: “Scrapy hangs if an exception raises in start_requests”
prefer “Fix hanging when exception occurs in start_requests (#411)”
instead of “Fix for #411”. Complete titles make it easy to skim through
the issue tracker.

Finally, try to keep aesthetic changes (PEP 8 compliance, unused imports
removal, etc) in separate commits than functional changes. This will make pull
requests easier to review and more likely to get merged.

Don’t use docstrings for documenting classes, or methods which are
already documented in the official (sphinx) documentation. Alternatively,
do provide a docstring, but make sure sphinx documentation uses
autodoc extension to pull the docstring. For example, the
ItemLoader.add_value() method should be either
documented only in the sphinx documentation (not it a docstring), or
it should have a docstring which is pulled to sphinx documentation using
autodoc extension.

Do use docstrings for documenting functions not present in the official
(sphinx) documentation, such as functions from scrapy.utils package and
its sub-modules.