NAME

geo-latlon2place-makedb - generate database for use with Geo::LatLon2Place

SYNOPSIS

geo-latlon2place-makedb [OPTION]... inputfile.txt outputfile.cdb

DESCRIPTION

abc

OPTIONS AND ARGUMENTS

geo-latlon2place-makedb requires two arguments: a text file with geo data and the name of the database file to be written.

By default, the input file is considered to be in geonames gazetteer format, but this can be customized using --extract.

--cellsize km (default 20)

The (minimum) size of a single grid cell in km - the data is binned into cells of at least this size. It should not be larger than the search radius.

--extract geonames|geonames-postalcodes|perl...

The extraction method: the default is geonames, which expects a geonames database (https://download.geonames.org/export/dump/, for example DE.txt, cities500.txt or allCountries.txt) and extracts placename, countrycode strings from it.

The method geonames-postalcodes (not yet implemented) does the same, but for a geonames postal code database https://download.geonames.org/export/zip.

Lastly, you can specify a perl fragment that implements your own filtering and extraction.

FILTERING AND EXTRACTION

Record selection and the returned data are not fixed - you can filter your records yourself and associate any (reasonably short, the maximum is a bit above 200 octets) binary blob with a coordinate pair.

To do this, you have to provide a perl fragment that extracts latitude, longitude, a weight and the associated data blob from an input line stored in $_. The file is opened using the :perlio layer, so if your input file is in UTF-8, so will be $_.

For example, the following would expect an input file with space separated latitude, longitude, weight and name, where name can contain spaces, which is useful when you wat to provide your own input data:

geo-latlon2place-makedb --extract 'chomp; split / /, 4' input output

A slighly more verbose example expecting only latitude, longitude and a name would be:

geo-latlon2place-makedb --extract '
   chomp;
   my ($lat, $lon, $name) = split / /, 4;
   ($lat, $lon, 1, $name)
' input output

If you want to skip certain lines without adding anything to the database, you can return nothing:

geo-latlon2place-makedb --extract '
   chomp;
   my ($lat, $lon, $name) = split / /;
   return unless $lat < 0; # only add southern hemisphere points
   ($lat, $lon, 1, $name)
' input output

In general, the fragment should return either an empty list, or a four-tuple with decimal latitude, longitude, a weight (integer 0..255) and the binary data to be associated with the coordinate. Other than the weight, these should be self-explaining. The weight is used during search and will be multiplied to the square of the distance, and is used to make larger cities win over small ones when the coordinate is somewhere between them.

The geonames filter looks similar to this fragment, which shows off more possibilities:

my ($id, $name, undef, undef, $lat, $lon, $t1, $t2, $cc, undef, $a1, $s2, $a3, $a4, $pop, undef) = split /\t/;

return if $t1 ne "P"; # only places

# minimum population 200, or it is a populated place with no known population
$pop => 200
   or ($pop eq "" and $t2 eq "PPL")
   or return;

# skip certain places we are not interested in
return if $t2 eq "PPLX"; # section of populated place
return if $t2 eq "PPLW"; # destroyed populated place
return if $t2 eq "PPLH"; # historical populated place
return if $t2 eq "PPLQ"; # abandoned populated place

# geonames has a lot of very long place names which aren't
# actually place names, so ignore very long names
60 > length $name
   or return;

# we estimate a weight by dividing 25 by the radius of the place,
# which we get by assuming a fixed population density of 5000 people/km²,
# which is almost always a considerable over-estimate.
# 25 and 5000 are pretty much made-up, feel free to improve and
# send me the results.
my $w = 25 / (1 + sqrt $pop / 5000);

# administrative centers get a fixed low weight
if ($t2 =~ /^PPLA(\d*)/) {
   $w = $1 || 1;
}

($lat, $lon, $w, "$name, $cc")

AUTHOR

Marc Lehmann <schmorp@schmorp.de>