/ 
WebFetch-0.06
River stage one • 4 direct dependents • 4 total dependents
/ WebFetch::General

NAME

WebFetch::General - download and save headlines from other WebFetch sites

SYNOPSIS

In perl scripts:

use WebFetch::General;

From the command line:

perl -w -MWebFetch::General -e "&fetch_main" -- --dir directory --file output-filename --url source-url [--format format]

DESCRIPTION

This module gets the current headlines from any WebFetch site that exports its news with the "WebFetch Export" format. You can do this with the --export command-line parameter. It works for any WebFetch module that defines the export() function, which includes all the modules that come packaged with WebFetch.

The webmaster of a remote site only needs to arrange for a cron job to update a WebFetch Export file, and let others know the URL to reach that file. (On the exporting site, it is most likely they'll use WebFetch::SiteNews to export their own news.) Then you can use the WebFetch::General module to read the remote file and generate and HTML summary of the news.

After WebFetch::General runs, the file specified in the --file parameter will be created or replaced. If there already was a file by that name, it will be moved to a filename with "O" (for old) prepended to the file name.

FORMAT STRINGS

WebFetch::General uses a format string to generate HTML from the incoming data. The default format for retrieved data is

<a href="%url%">%title%</a>

This means that fields named "url" and "title" must exist in the incoming WebFetch-exported data, and will be used to fill in the %url% and %title% strings, respectively. You may use the --format parameter to specify any format you wish. But the field names you choose in the format must match fields defined in the input stream. Otherwise they will fail to be expanded.

THE "WebFetch Export" FILE FORMAT

This is an example WebFetch Export file generated by WebFetch::SiteNews:

[WebFetch export]
Version: 0.02
# This was generated by the Perl5 WebFetch module.
# WebFetch info can be found at http://www.svlug.org/sw/webfetch/
#
# Exported from WebFetch::SiteNews
# "date" is the date of the news
# "title" is a one-liner title
# "url" is url to the news source
# "text" is the news text

date: March 21, 1999
title: WebFetch 0.03 beta released
url: http://www.svlug.org/sw/webfetch/news.shtml#19990321-000
text:   &<;a href="WebFetch-0.03.tar.Z"&>;WebFetch 0.03 beta&<;/a&>; released

date: January 11, 1999
title: WebFetch 0.01 beta released
url: http://www.svlug.org/sw/webfetch/news.shtml#19990111-000
text:   &<;a href="WebFetch-0.01.tar.Z"&>;WebFetch 0.01 beta&<;/a&>; released

Each news item is separated by a blank line.

Within each news item, the fields use a "name: value" format, similar to RFC822 headers (i.e. as in e-mail and news.)

The names of the fields are chosen by the exporting module. Though for the convenience of the user, the author of an exporting module should keep in mind the default WebFetch::General format uses fields called "url" and "title". If you use fields by different names, warn your receiving users that they will need to make a format to use with WebFetch::General, though you may provide them with one in their setup instructions.

AUTHOR

WebFetch was written by Ian Kluft for the Silicon Valley Linux User Group (SVLUG). Send patches, bug reports, suggestions and questions to webfetch-maint@svlug.org.

SEE ALSO

WebFetch