OpenStreetMap Nominatim Server for Geocoding

5.00 avg. rating (98% score) - 9 votes

Here’s how to install the OpenStreetMap Nominatim service on your own server. It can be used to geocode and reverse geocode addresses and map coordinates. You will also get a web interface which loads map tiles from openstreetmap.org while doing geocoding requests using your own server.

I was faced with thousands of geocoding requests per hour, and with such a high volume it is not wise to burden the global OSM servers. Even with aggressive caching it would have potentially been too much.

I will use Ubuntu 14.04 LTS as the platform. Just a basic install with ssh server. We will install Apache to serve http requests. Make sure you have enough disk space and RAM to hold the data and serve it efficiently. I used the Finland extract, which was about a 200 MB download. The resulting database was 26 GB after importing, indexing and adding Wikipedia data. The Wikipedia data probably actually took more disk space than the OSM data. My server has 4 GB RAM, which seems to be enough for this small data set.

Sofware requirements

PostgreSQL with PostGIS extension:

Apache with PHP5:

Git and various tools:

Tool for handling OpenStreetMap data:

Needed libraries:

Kernel tuning

We must increase kernel shared memory limits. Also we’ll reduce swappiness and set the kernel memory overcommit to be a bit more conservative. Make sure you have anough swap space, more than physical memory is a good idea.

These have been estimated using 4 GB RAM.

PostgreSQL tuning

The following directives were set in /etc/postgresql/9.3/main/postgresql.conf:

The buffer and memory values depend on your available RAM so you should set sane values accordingly.  These are good for 4 GB RAM.

There are two truly dangerous settings here. Setting fsync off will cause PostgreSQL to report successful commits before the disk has confirmed the data has been written successfully. Setting full_page_writes off exposes the database to the possibility of partially updated data in case of power failure. These options are used here only for the duration of the initial import, because setting them so makes the import much faster. Fsync and full page writes must be turned back on after the import to ensure database consistency.

The synchronous_commit also jeopardizes the durability of committed transactions by reporting successful commits to clients asynchronously, meaning that the transaction may not have been fully written to disk. It does not compromize  consistency as the two previous directives discussed. It just means that in the case of power failure, some recent transactions that were already reported to the client as having been committed, may in fact be aborted and rolled back. The database will still be in a consistent state. We’ll leave it off because it speeds up the queries, and we don’t really care about it in this database. We can always rebuild it from scratch. But if you have other important databases in the same database cluster, I would recommend turning it back on as well, since the setting is cluster-wide.

The maintenance_work_mem will also be reduced to a lower value later, after the import.

Restart PostgreSQL to apply changes:

Dedicated user

We’ll install the software under this user’s home directory (user id “nominatim”).

Database users

We will also make “nominatim” a database user, as well as the already created “www-data” (created by the apache2 package).

Nominatim will be a superuser and www-data will be a regular one.

This must be done as a database administrator. The “postgres” user by default is one (root is not). Change user:

Create database users:

Download and Compile Nominatim

Su to user nominatim:

Set some environment variables. We’ll use these later. These are the download locations for files and updates. Lines 3 and 4 below are for Finland. Customize to your needs.

Also set the BASE_URL to point to your server and install directory. The last forward slash seems to be important.

You can browse available areas at download.geofabrik.de.

Clone the Repository from Github

We will use the release 2.3 branch:

The –recursive option is needed to clone everything, because the repository contains submodules.

Compile Nominatim

Simply:

If there were no errors, we are good. In case of missing libraries, check that you installed all requirements earlier.

Configure Local Settings

Some required settings should go to ~/Nominatim/settings/local.php:

The last four definitions are only required if you are going to do incremental updates.

Download Data

OpenStreetMap data (about 200 MB for Finland):

The OSM data is all that is really required. The rest below are optional.

Wikipedia data (about 1.4 GB):

Special phrases for country codes and names (very small):

Special search phrases (a few megabytes):

Next we’ll import all this stuff into the database.

Import Data

The utils/setup.php will create a new database called “nominatim” and import the given .pbf file into it. This will take a long time depending on your PostgreSQL settings, available memory, disk speed and size of dataset. The full planet can take days to import on modern hardware. My small dataset took a bit over two hours.

The messages will be saved into setup.log in case you need to look at them later.

If you had the Wikipedia data downloaded, the setup should have imported that automatically and told you about it. You can import the special phrases data if you downloaded it earlier using these commands:

Database Production Settings

Now that the import is done, it is time to configure the database to settings that are suitable for production use. The following changes were made in /etc/postgresql/9.3/main/postgresql.conf:

You should also set the synchronous_commit directive to on if you have other databases running on this same database cluster. See the PostgreSQL Tuning section earlier in this post.

Apply changes:

Create the Web Site

The following commands have to be run as root.

Create a directory to install the site into and set permissions:

Ask bots to keep out:

Apache configuration

Edit the default site configuration file /etc/apache2/sites-enabled/000-default.conf and make it look like something like below:

Apply changes by restarting web server:

Install the Nominatim Web Site

The installation should be done as the nominatim user:

Run the setup.php with the option to create the web site:

At this point, the site is ready and you can point your browser to the base URL and try it out. It should look something like this:

Valinta_064

Enable OSM Updates

We will configure a cron job to have the database updated periodically using diffs from Geofabrik.de. As the nominatim user:

Enable hierarchical updates in the database (these were off during import to speed things up):

Run once to get up to date:

There seems to be a time check in the import function that prevents another run immediately. Instead it will wait until the interval configured in the “Configure Local Settings” section earlier in this post has passed (default is 1 day).

We will add the command to crontab to be executed every monday at 03:00. Run:

Add a new line to the crontab file looking like this:

A log file will be created as /home/nominatim/nominatim-update.log.

Wrap Up

That’s it! You can get JSON or XML data out of the database with http requests. A couple of examples:

Forward geocoding:

  • http://maps.example.org/nominatim/search.php?format=jsonv2&q=mannerheimintie, helsinki

Reverse geocoding:

  • http://maps.example.org/nominatim/reverse.php?format=jsonv2&lat=60.1805&lon=24.9434

Check out the Nominatim documentation for details.

This post was adapted from my notes of such an installation process. Let me know in the comments section if I introduced any mistakes. Other comments are also welcome!

Useful links

33 thoughts on “OpenStreetMap Nominatim Server for Geocoding”

  1. Hello Kortsi, great article.

    When I try to access the reverse.php get this: “{” error “:” Unable to geocode “}”. I followed all the steps of your article, which can be?

    Thanks for your attention.

    1. That may mean there’s something wrong with your query. Did you give the “lat” and “lon” parameters? Those are required.

  2. i have an erro in import data to postgres

    Using projection SRS 4326 (Latlong)
    NOTICE: table “place” does not exist, skipping
    NOTICE: type “keyvalue” does not exist, skipping
    NOTICE: type “wordscore” does not exist, skipping
    NOTICE: type “stringlanguagetype” does not exist, skipping
    NOTICE: type “keyvaluetype” does not exist, skipping
    NOTICE: function get_connected_ways(pg_catalog.int4[]) does not exist, skipping
    Allocating memory for dense node cache
    Allocating dense node cache in one big chunk
    Out of memory for dense node cache, reduce –cache size
    Error occurred, cleaning up
    ERROR: Error executing external command: /home/nominatim/Nominatim/osm2pgsql/osm2pgsql -lsc -O gazetteer –hstore -C 1024 -P 5432 -d nominatim /home/nominatim/Nominatim/data/latest.osm.pbf
    Error executing external command: /home/nominatim/Nominatim/osm2pgsql/osm2pgsql -lsc -O gazetteer –hstore -C 1024 -P 5432 -d nominatim /home/nominatim/Nominatim/data/latest.osm.pbf

    1. I think you are running out of RAM because it says “Out of memory for dense node cache, reduce –cache size”. That’s why the whole import fails.

      You should try with a lower –osm2pgsql-cache size. Or increase swap size for the machine.

  3. i put my ubuntu server in a computer with a cpu of 3 core with 4 ram

    I solved my problem, it was the cache size, I put 1024
    then i have the problem
    ——
    Out of memory for dense node cache, reduce –cache size
    Error occurred, cleaning up
    ——
    i saw in a external post that i was the cache size, and i put 512 and it’s work….

    this is the post
    https://github.com/twain47/Nominatim/issues/105

    ——
    Yes, you do because osm2pgsql reports it is out of memory. It is currently trying to use 4GB and that seems to be too much. If you have 8GB of memory, try with 2048 and if that does not work either with 1024. Also make sure you have a few dozen GB of swap enabled.
    ——

    in URL: http://localhost/nominatim/
    i got map, but in top it showing error

    Data: DB Error: no such table
    ———————
    You don’t need to change anything in any files, just run ./utils/setup.php –osm-file –all –osm2pgsql-cache 1024 for the import as described in the installation instructions.
    ——————-

    the problem was solved
    http://i58.tinypic.com/2e3n4ed.png

  4. Setting postgresql’s shared_buffers to 2GB when you have only 4GB RAM is not the best of ideas because this memory will be permanently reserved by the database server. Postgres recommends to use around 25% of memory but something like 256MB is more to the point with so little RAM because there are other applications running next to postgres during import (notably osm2pgsql which hogs lots of memory).

    Also, don’t forbid to overcommit memory unless you have really a lot of RAM. osm2pgsql relies on being able to overcommit.

  5. Hello Kortsi, very interesting article.

    As I read it to install our Nominatim DB, I give some feedback on our experience.
    The server used is vm with 4*3.1 Ghz, 15 Go Ram and high speed harddrives of 400 Go.

    We imported only some country (fr, uk, it, be, de, sp, it). To process them we merged all files in a single one of 9 Go. It took 8 hours.
    The import ran for about 36 hours and consumed 180 Go of disk.

    Best Regards
    Al

  6. kortsi hello , good night!

    when I try to use reverse geocoding , Info Some of addressparts not appear at the moment only appears this for example :

    Return , BrazilRetornoBrazilbr

    You know what might be causing the lack of data ?

  7. Hi,

    I am at the importing phase and when I type “./utils/setup.php –osm-file data/latest.osm.pbf –all –osm2pgsql-cache 1024 2>&1 | tee setup.log” I get the error “the path supplied to –osm-file does not exist”

  8. Worked it out I had downloaded osn file to root “./utils/setup.php –osm-file latest.osm.pbf –all –osm2pgsql-cache 1024 2>&1 | tee setup.log” fixed it

  9. Muchas gracias por la excelente explicacion de como usar esta herramienta, voy a trabajar esta configuracion para OpenGTS. Saludos

  10. sorry – got a big problem. as a newbie i dont know much about apache

    after
    ./utils/setup.php –create-website /var/www/html/nominatim

    i got that error:

    WARNING: Unable to access the website at http://maps.example.org/nominatim/
    You may want to update settings/local.php with @define(‘CONST_Website_BaseURL’, ‘http://[HOST]/[PATH]/’);
    Setup finished.

  11. What more can I say but it works!

    The most complete and accurate set up instructions I have found!

  12. Hi Kortsi.
    This a best guide i ever read about Nominatim installation.
    Even this time some changes need to be done that different from what you said.
    Anyway, i am enjoying it, that’s great.
    Thank you so much.

    lieubv@gmail.com

  13. Excellent instructions – these are the first ones that actually have worked first time. I have imported a small sample country to ensure that the platform is working as expected.

    I will not be using the updating functionality. How do I now import another country? What is the command required?

    Thanks!!

  14. If I wanted to use say jtiledownloader to export tiles please could you tell me how?

    Thanks

    Andy.

  15. I am looking at servicing 100 reverse geocoding requests per second.

    Does anybody know what sort of machine could do this, or would multiple servers be required?

    I have this running on a Virtualbox VM with 4 cores and 4GB RAM but can only do about 10 per second.

    Thanks

    Andy.

  16. Hi guys,

    We have been trying to setup a server for quite some time now and are getting database errors every time. After the import into Postgres (the last step) it’s an error every singl time.

    Is there a way/place where I can get an indexed DB dump which I can import and skip the import process??

    Hope you can help (i will be watching this threat)

    1. I haven’t seen any DB dumps around. One issue might be size. I did a full planet import about a year ago and the resulting database took nearly a terabyte of disk space.

      What kind of an error are you getting?

  17. Hello, thanks for the information but i have a problem in import data to postgres:

    Using projection SRS 4326 (Latlong)
    NOTICE: table “place” does not exist, skipping
    NOTICE: type “keyvalue” does not exist, skipping
    NOTICE: type “wordscore” does not exist, skipping
    NOTICE: type “stringlanguagetype” does not exist, skipping
    NOTICE: type “keyvaluetype” does not exist, skipping
    NOTICE: function get_connected_ways(pg_catalog.int4[]) does not exist, skipping
    Allocating memory for dense node cache
    Out of memory for node cache dense index, try using “–cache-strategy sparse” instead
    Error occurred, cleaning up
    ERROR: Error executing external command: /home/nominatim/Nominatim/osm2pgsql/osm2pgsql -lsc -O gazetteer –hstore -C 256 -P 5432 -d nominatim /home/nominatim/Nominatim/data/latest.osm.pbf
    Error executing external command: /home/nominatim/Nominatim/osm2pgsql/osm2pgsql -lsc -O gazetteer –hstore -C 256 -P 5432 -d nominatim /home/nominatim/Nominatim/data/latest.osm.pbf

    PD:I have 6 GB of RAM And lower the size of cache-size up to 256 MB but I do not solve my problem.

  18. This is a nice piece work and I have used for about 5 years now on my Ubuntu16 box, now I want to upgrade from Nominatim 2.3 to 4xx. Do you know any good sites that deals with this migration please?
    Thanks

Leave a Reply