5.15.4.1. Building your own Nominatim Server

<< Click to Display Table of Contents >>

Navigation:  5. Detailed description of the Actions > 5.15. Operational Research > 5.15.4. Geocode using Nominatim >

5.15.4.1. Building your own Nominatim Server

 
We copied the majority of the steps from the Nominatim install guide and we modified slightly where we got blocked (so if this document gets outdated, please visit https://nominatim.org/release-docs/latest/admin/Installation/) for the updated steps.

 

Here we will follow the steps for a Ubuntu 20  installation.

 
The recommended mininum HW requirements are:

-HardDisk : +/- 1TB  ( NVME disks is recommended)

-RAM : 40GB for EU  (64GB for global coverage)

-Fast CPU

-Ubuntu 18 or 20

 

The startpoint is to have an “empty” ubuntu box (inside a VM or not)

 

5.15.4.1.1. Installing the Required Software

 

These instructions expect that you have a freshly installed Ubuntu 20.04.

Make sure all packages are are up-to-date by running:

 

sudo apt update -qq

 

Now you can install all packages needed for Nominatim:

 

sudo apt install -y php-cgi
sudo apt install -y build-essential cmake g++ libboost-dev libboost-system-dev \
     libboost-filesystem-dev libexpat1-dev zlib1g-dev \
     libbz2-dev libpq-dev libproj-dev \
     postgresql-server-dev-12 postgresql-12-postgis-3 \
     postgresql-contrib-12 postgresql-12-postgis-3-scripts \
     php php-pgsql php-intl libicu-dev python3-dotenv \
     python3-psycopg2 python3-psutil python3-jinja2 python3-icu

 
 

5.15.4.1.2. System Configuration

 

The following steps are meant to configure a fresh Ubuntu installation for use with Nominatim. You may skip some of the steps if you have your OS already configured.

 

1. Creating Dedicated User Accounts

 

Nominatim will run as a global service on your machine. It is therefore best to install it under its own separate user account. In the following we assume this user is called nominatim and the installation will be in /srv/nominatim. To create the user and directory run:

 

sudo useradd -d /srv/nominatim -s /bin/bash -m nominatim

 

To be able to copy and paste instructions from this manual, export user name and home directory now like this:

 

export USERNAME=nominatim
export USERHOME=/srv/nominatim

 

Never, ever run the installation as a root user. You have been warned.

Make sure that system servers can read from the home directory:

 

chmod a+x $USERHOME

 
 

5.15.4.1.3. Setting up & Tune PostgreSQL

 

You might want to tune your PostgreSQL installation so that the later steps make best use of your hardware. You should tune the following parameters in your postgresql.conf file.
 

(postgresql.conf is for ubuntu & postgres 12 in /etc/postgres/12/main/ )

 

shared_buffers = 2GB
maintenance_work_mem = 10GB
autovacuum_work_mem = 2GB
work_mem = 50MB
effective_cache_size = 24GB
synchronous_commit = off
max_wal_size = 1GB
checkpoint_timeout = 10min
checkpoint_completion_target = 0.9

 

The numbers above seem to work fine for 64GB RAM machine. Adjust to your setup (i.e. make sure you have enough RAM!). A higher number for max_wal_size means that PostgreSQL needs to run checkpoints less often but it does require additional space on your disk.

 

Restart the postgresql service after updating the above config file:

 

sudo systemctl restart postgresql

 

Finally, we need to add two postgres users: One for the user that does the import and another for the webserver which should access the database for reading only:

 

sudo -u postgres createuser -s $USERNAME
sudo -u postgres createuser www-data

 
 

5.15.4.1.4. Installing the Nominatim code

 

1. Building and Configuration

 

Get the source code for the release and unpack it:

 

cd $USERHOME
wget https://nominatim.org/release/Nominatim-3.7.1.tar.bz2
tar xf Nominatim-3.7.1.tar.bz2

 

The code must be built in a separate directory. Create this directory, then configure Nominatim in there:

 

mkdir $USERHOME/build
cd $USERHOME/build
cmake $USERHOME/Nominatim-3.7.1
make
sudo make install

 

 
2. Setting up a webserver (Apache)

 

The webserver should serve the php scripts from the website directory of your project directory. Therefore, set up a project directory and populate the website directory:

 

mkdir $USERHOME/nominatim-project
cd $USERHOME/nominatim-project
nominatim refresh --website

 

Apache has a PHP module that is reauired for Nominatim. To install this module run:

 

sudo apt install -y apache2 libapache2-mod-php

 

You need to create an alias to the website directory in your apache configuration. Add a separate nominatim configuration to your webserver:
 

sudo tee /etc/apache2/conf-available/nominatim.conf << EOFAPACHECONF
<Directory "$USERHOME/nominatim-project/website">
 Options FollowSymLinks MultiViews
 AddType text/html   .php
 DirectoryIndex search.php
 Require all granted
</Directory>
 
Alias /nominatim $USERHOME/nominatim-project/website
EOFAPACHECONF

 
Then restart apache:

 

sudo a2enconf nominatim
sudo systemctl restart apache2

 

The Nominatim API is now available at http://localhost/nominatim/.

 
 

5.15.4.1.5. Importation 1: Creating the Project directory

 

Before you start the import, you should create a project directory for your new database installation. This directory receives all data that is related to a single Nominatim setup: configuration, extra data, etc. Create a project directory apart from the Nominatim software and change into the directory:

 

mkdir ~/nominatim-project
cd ~/nominatim-project

 

In the following, we refer to the project directory as $PROJECT_DIR. To be able to copy&paste instructions, you can export the appropriate variable:

 

export PROJECT_DIR=~/nominatim-project

 
The Nominatim tool assumes per default that the current working directory is the project directory but you may explicitly state a different directory using the --project-dir parameter. The following instructions assume that you run all commands from the project directory.

 
 

5.15.4.1.6. Importation 2: Configuration setup in .env

 
The Nominatim server can be customized via an .env configuration file in the project directory. This is a file in dotenv format which looks the same as variable settings in a standard shell environment. You can also set the same configuration via environment variables. All settings have a NOMINATIM_ prefix to avoid conflicts with other environment variables.
 

cd  $PROJECT_DIR
touch .env

 
 

5.15.4.1.7. Importation 3: Enable Flatnode storage

 
If you plan to import a large dataset (e.g. Europe, North America, planet), you should also enable flatnode storage of node locations. With this setting enabled, node coordinates are stored in a simple file instead of the database. This will save you import time and disk storage. Add to your .env:
 

NOMINATIM_FLATNODE_FILE="/path/to/flatnode.file"

 
Replace the second part with a suitable path on your system and make sure the directory exists. There should be at least 75GB of free space.

 
 

5.15.4.1.8. Importation 4: Additional optional download: Wikipedia/Wikidata rankings

 

Wikipedia can be used as an optional auxiliary data source to help indicate the importance of OSM features. Nominatim will work without this information but it will improve the quality of the results if this is installed. This data is available as a binary download. Put it into your project directory:
 

cd $PROJECT_DIR
wget https://www.nominatim.org/data/wikimedia-importance.sql.gz

 
The file is about 400MB and adds around 4GB to the Nominatim database. We didn’t perform this step to create the VM that contains the “ready to be used” nominatim server.

 
 

5.15.4.1.9. Importation 5: Additional optional download: Great Britain& USA postcodes

 

Nominatim can use postcodes from an external source to improve searches that involve a GB or US postcode. This data can be optionally downloaded into the project directory:
 

cd $PROJECT_DIR
wget https://www.nominatim.org/data/gb_postcode_data.sql.gz
wget https://www.nominatim.org/data/us_postcode_data.sql.gz

 
 

5.15.4.1.10. Importation 6: Get and load the map data

 

If you only need geocoding for a smaller region, then precomputed OSM extracts are a good way to reduce the database size and import time. Geofabrik offers extracts for most countries. They even have daily updates which can be used with the update process described in this webpage. There are also other providers for extracts. Please be aware that some extracts are not cut exactly along the country boundaries. As a result, some parts of the boundary may be missing which means that Nominatim cannot compute the areas for some administrative areas.

 

You can now download the data extract for the region of your interest:
 

You can find the continents here : https://download.geofabrik.de/

Or for EU countries : https://download.geofabrik.de/europe.html

 

Download the raw osm.pbf ( or download the zipped version and unzip )
THIS WILL BE THE <<DATAFILE>> for the next step

 

Issue the following command from the project directory to start the import (The official guide says to start from the build directory but this did not work):

 

nominatim import --osm-file <data file> 2>&1 | tee setup.log

 

Warning: The above step takes more than 48 hours!

 
 

clip1022

If you get a database error your user can’t access the data à add the users :

 Sudo su postgres

 Psql

 Create user <<your user name>>:

  grant all privileges on database nominatim to <<your user name>>;

  \q

 Exit

 

If the import fails and you want to retry, it is easier to remove the Nominatim database:

  Sudo su postgres

  Psql

  Drop database nominatim

 

 
5.15.4.1.11. Importation 7: Notes on full planet imports

 

Even on a perfectly configured machine the import of a full planet takes around 2 days. Once you see messages with Rank .. ETA appear, the indexing process has started. This part takes the most time. There are 30 ranks to process. Rank 26 and 30 are the most complex. They take each about a third of the total import time. If you have not reached rank 26 after two days of import, it is worth revisiting your system configuration as it may not be optimal for the import.

 

 
5.15.4.1.12. Importation 8: Testing the installation

 

Run this script to verify all required tables and indices got created successfully:

 

nominatim admin --check-database

 

Now you can try out your installation by running:
 

nominatim serve

 
This runs a small test server normally used for development. You can use it to verify that your installation is working. Go to http://localhost:8088/status.php and you should see the message OK. You can also run a search query, e.g. http://localhost:8088/search.php?q=Berlin.

 

If you are using the “read to use” VM, you can test it by running inside a command-prompt from the VM-host (e.g. from a Windows command-prompt):
 

curl -kg http://192.168.23.129:8081/nominatim/status.php?format=json -X GET

 
 

5.15.4.1.13. Enable bulk api requests.

 
Edit the file “website/search.php” inside the Project directory (i.e. edit the file “/srv/nominatim/nominatim-project/website/search.php”) and replace the following string:
 

@define(‘CONST_search_batchMode’, ‘false’);

 
…with:
 

@define(‘CONST_search_batchMode’, ‘true’);