What if you wanted to do something like: process "tag.class", key => 'TEXT', name => '@id'; # multiple variable creation -- this does work OR process "tag.class", key => 'TEXT'.substr(3,6); # modifying the data instead of copying the exact contents of specified attribute -- I have not been able to find any way to do this

Am I just not experienced enough, and any competent Perl programmer would know all the syntax they could use and modifications they could do to the sparse examples and explanations given in the documentation? Am I just not looking in the right place for Perl documentation? Or do some Perl modules just not have good documentation and reading the source code and experimentation is necessary?

I've found documentation across CPAN to be generally clear and complete. Web::Scraper does not expand into much detail, but there is enough to experiment with. It also states that "There are many examples in the eg/ dir packaged in this distribution. It is recommended to look through these". Documentation should provide enough information to support its reader's requirements without them having to study the source. If you are competent enough to study the source, then you will inevitably develop a deeper understanding of the module and its limitations. Its also a good idea to look at the modules dependencies, in the instance of Web::Scraper, HTML::TreeBuilder::XPath and HTML::Selector::XPath appear to handle xpath expressions therefore may provide additional syntax / documentation / examples. If this is the first time you've approached web scraping in Perl, although Web::Scraper has been designed to simplify the process, it would be good to research into "rawer" techniques, which give you more control at every stage of the scraping process i.e. HTML::Element.

This is pretty much my very beginnings of Perl use, and have gone with bare minimums for a deeper understanding of Perl for most of my project, but this project really needs the depth of css selector syntax to do right so I am planning on sticking with Web::Scrapper.

I still do not understand why the syntax of this package is this way but have experimented and read enough examples to get a far better understanding of at least some of what you can do with it. It is worth mentioning that you can nest these "process"es. So for example if you are scraping a table, you can do:

It appears that your knowledge of Web::Scraper has greatly improved since your original post.

In answer to your query "I still do not understand why the syntax of this package is this way", the module does not construct its object in the conventional manner. In Perl, there are many ways to achieve equal outcomes, do not necessarily regard Web::Scraper as a model example.

Remember that Web::Scraper is basically an interface to more extensive classes, with the benefit of simplifying the core scraping process for the programmer. In your instance, it uses HTML::Selector::XPath to parse css selectors into an xpath, usable by HTML::TreeBuilder::XPath, which does the initial html parsing. If Web::Scraper doesn't do what you need directly due to limitations, use these and other classes to do what you need indirectly.