Difference between revisions of "Nominatim for Traffic Accident Database"

Revision as of 13:45, 4 March 2015

>> return Cedeus_IDE

Objective

Develop a script for automated geo-referencing of Chiles transit accident database.

Tools

• Nominatim
• MS-Access / LibreOffice : need to load the MS Access DB from Carabineros de Chile in OpenOffice/LibreOffice: http://askubuntu.com/questions/187389/is-it-possible-to-open-an-access-2010-database-file-without-using-wine-or-virtua/519571#519571
• iPython + Notebook

Installing Nominatim

• Nominatim installation manual: https://wiki.openstreetmap.org/wiki/Nominatim/Installation

Setup of Nominatim VM

1. copying basicubunutu1404.vdi file and renaming to nominatim.vdi
2. creating the nominatim VM on CedeusDB (ip.18):
   • VBoxManage createvm --name nominatim --ostype Ubuntu_64 --register
   • VBoxManage modifyvm nominatim --memory 4096
   • VBoxManage modifyvm nominatim --cpus 4
   • VBoxManage modifyvm nominatim --nic1 nat
   • VBoxManage storagectl nominatim --name "SATA Controller" --add sata --controller IntelAhci
   • assign the (old) disk image a new uuid

     VBoxManage internalcommands sethduuid nominatim.vdi

   • attach the (old) disk image:

     VBoxManage storageattach "nominatim" --storagectl "SATA Controller" --port 0 --device 0 --type hdd --medium nominatim.vdi

   • VBoxManage storagectl nominatim --name "IDE Controller" --add ide --controller PIIX4
   • set the nat rules (ports):

     VBoxManage modifyvm nominatim --natpf1 "ssh,tcp,,20022,,22"

     VBoxManage modifyvm nominatim --natpf1 "apache,tcp,,20080,,80"

3. optional - set VRDE port:
   • VBoxManage modifyvm nominatim --vrdeport 7761 (Note: 3389 is the default port anyway)
4. check all settings with VBoxManage showvminfo nominatim
5. start the VM

   VBoxHeadless -s nominatim --vrde on &

   the VM will listen on port 7761 - connect with GTKvncViewer to check

   • check if ssh connection works
6. change the computers name in /etc/hosts and /etc/hostname
7. restarting Apache gives the error message: "apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 ..."

   => solve this by adding ServerName nominatim as the last line in /etc/apache2/apache2.conf file

8. shut the VM down to resize the HDD if necessary
   • check the HDD size: VBoxManage showhdinfo nominatim.vdi returns:

UUID:           e148d758-00ac-4b60-a067-d5c97316069d
Parent UUID:    base</br>
State:          created
Type:           normal (base)
Location:       /home/ssteinig/nominatim.vdi
Storage format: VDI
Format variant: dynamic default
Capacity:       100000 MBytes
Size on disk:   6224 MBytes
In use by VMs:  nominatim (UUID: be795392-3786-4f3b-9714-a9445f91855d)

• so I resized the disk to 200GB as, according to Nominatim infos the disk size for a full planet is 700GB

  VBoxManage modifyhd nominatim.vdi --resize 200000

Setup of Nominatim itself

Install needed libraries

• install PHP 5
• install Postgres
• install PEAR
• install GCC, GIT, protbuf, wget, etc.
• install Osmosis

=> see https://wiki.openstreetmap.org/wiki/Nominatim/Installation#Software

sudo apt-get install build-essential libxml2-dev libgeos-dev libpq-dev libbz2-dev libtool automake libproj-dev
sudo apt-get install gcc proj-bin libgeos-c1 git osmosis libgeos++-dev
sudo apt-get install php5 php-pear php5-pgsql php5-json php-db
instead of: >> sudo apt-get install postgresql postgis postgresql-contrib postgresql-9.1-postgis postgresql-server-dev-9.1
  use: >> sudo apt-get install postgresql postgis postgresql-contrib postgresql-9.3-postgis-2.1 postgresql-server-dev-9.3
sudo apt-get install libprotobuf-c0-dev protobuf-c-compiler

and perhaps

sudo apt-get install git autoconf-archive

Edit PostreSQL settings

perhaps edit Postgres settings (before data import) sudo nano /etc/postgresql/9.3/main/postgresql.conf

=> see https://wiki.openstreetmap.org/wiki/Nominatim/Installation#Software

My machine has only 4GB, so I changed:

• shared_buffers = 128MB => 512MB
• work_mem = 50MB (previously uncommented, and set to 1MB)
• maintenance_work_mem = 2GB (previously commented, and set to 16MB) //=> This value should be reduced again afterwards to avoid swapping when autovacuum runs.
• effective_cache_size = 2GB (previously commented, and set to 128MB)
• synchronous_commit = off (previously commented, and set to on)
• checkpoint_segments = 100 (previously commented, and set to 3)
• checkpoint_timeout = 10min (previously commented, and set to 5min)
• checkpoint_completion_target = 0.9 (previously commented, and set to 0.5)

and for initial import only:

• fsync = off (previously commented, and set to 'on')
• full_page_writes = off (previously commented, and set to 'on')

=> switch both values back to 'on' after import, to avoid database corruption.

Note, I did a restart of the postgresql service after to have the settings applied.

First Nominatim Installation

Get the source code

I used the last stable release 2.3.1:

wget http://www.nominatim.org/release/Nominatim-2.3.1.tar.bz2
tar xvf Nominatim-2.3.1.tar.bz2

and rename

mv Nominatim-2.3.1 Nominatim

Compiling

now compile the software

cd Nominatim
./configure
make

Note, the warning about missing lua libraries can be ignored. Nominatim does not make use of osm2pgsql's lua extension.