# NAME

Scrappy - All Powerful Web Spidering, Scrapering, Crawling Framework

# VERSION

version 0.9111110

# SYNOPSIS

    #!/usr/bin/perl
    use Scrappy;

    my  $scraper = Scrappy->new;
        $scraper->crawl('search.cpan.org',
            '/recent' => {
                '#cpansearch li a' => sub {
                    print $_[1]->{href}, "\n";
                }
            }
        );

# DESCRIPTION

Scrappy is an easy (and hopefully fun) way of scraping, spidering, and/or
harvesting information from web pages, web services, and more. Scrappy is a
feature rich, flexible, intelligent web automation tool.

Scrappy (pronounced Scrap+Pee) == 'Scraper Happy' or 'Happy Scraper'; If you
like you may call it Scrapy (pronounced Scrape+Pee) although Python has a web
scraping framework by that name and this module is not a port of that one.

# METHODS

## crawl

The crawl method is very useful when it is desired to crawl an entire website or
at-least partially, it automates the tasks of creating a queue, fetching and
parsing html pages, and establishing simple flow-control. See the SYNOPSIS for
a simplified example, ... the following is a more complex example.

    my  $scrappy = Scrappy->new;
    

    $scrappy->crawl('http://search.cpan.org/recent',
        '/recent' => {
            

            '#cpansearch li a' => sub {
                my ($self, $item) = @_;
                # follow all recent modules from search.cpan.org
                $self->queue->add($item->{href});
            }
            

        },
        '/~:author/:name-:version/' => {
            

            'body' => sub {
                my ($self, $item, $args) = @_;
                

                my $reviews = $self
                ->select('.box table tr')->focus(3)->select('td.cell small a')
                ->data->[0]->{text};
                

                $reviews = $reviews =~ /\d+ Reviews/ ?
                    $reviews : '0 reviews';
                

                print "found $args->{name} version $args->{version} ".
                    "[$reviews] by $args->{author}\n";
                

            }
            

        }
    );

# AUTHOR

Al Newkirk <awncorp@cpan.org>

# COPYRIGHT AND LICENSE

This software is copyright (c) 2010 by awncorp.

This is free software; you can redistribute it and/or modify it under
the same terms as the Perl 5 programming language system itself.