taskMapOSMatic - Tasks: task #10148, Switch to osmosis for DB updates

 
 

task #10148: Switch to osmosis for DB updates

Submitter:  David Decotigny <daviddecotigny>
Submitted:  Wed 03 Feb 2010 12:59:33 PM UTC
   
 
Should Start On:  Wed 03 Feb 2010 12:00:00 AM UTC Should be Finished on:  Fri 06 Aug 2010 10:00:00 PM UTC
Category:  database Priority:  * 7 - High
Status:  Done Privacy:  Public
Assigned to:  mpetazzoni Percent Complete:  100%
Open/Closed:  Closed Release:  None
Effort:  0.00
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 02 Nov 2010 08:56:18 AM UTC, comment #3: 

Waouh, I didn't know this. Great news !

David Decotigny <daviddecotigny>
Group administrator
Fri 29 Oct 2010 09:08:58 PM UTC, comment #2: 

This is done. The GIS database now hosted on a separate server uses 15-min update cycles with Osmosis.

Maxime Petazzoni <mpetazzoni>
Group administrator
Tue 13 Jul 2010 12:37:48 PM UTC, comment #1: 

The new server will use Osmosis for minutely updates. Migration is in progress, but hardware issues on the VM's host are making things difficult for now.

Maxime Petazzoni <mpetazzoni>
Group administrator
Wed 03 Feb 2010 12:59:33 PM UTC, original submission:  

From Thomas:
«
Les diff hourly et minute ont été supprimés, mais ils ont été
visiblement remplacés par une nouvelle méthode de synchronisation,
basée sur Osmosis:

 http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage#Replication_Tasks

Les données de différences sont apparemment dispo sur:
 http://planet.openstreetmap.org/hour-replicate/
 http://planet.openstreetmap.org/minute-replicate/

D'après ce que je comprends de
http://lists.openstreetmap.org/pipermail/osmosis-dev/2009-November/000310.html,
ce nouveau système de diff a l'air d'utiliser directement des
transactions PostgreSQL. Je suspecte que ce soit potentiellement plus
efficace que la méthode actuelle. À étudier, peut-être demander sur la
liste si il y a des compétences dans le domaine.

A+
»

Looks like osmosis is promising. And, in the short term, we will HAVE to use it.

Never looked deep enough into it, but it reveals an interesting pipeline-based architecture, even though osm2pgsql still seems to be necesary to update the DB (but maybe I'm missing something).

Anyway, this could be of interest for us, too:
http://lists.openstreetmap.org/pipermail/osmosis-dev/2009-November/000313.html
Especially this script: https://svn.toolserver.org/svnroot/mazder/diff-import/load-next

David Decotigny <daviddecotigny>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mpetazzoni (Posted a comment)
  • -email is unavailable- added by daviddecotigny (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-10-29 mpetazzoni StatusIn Progress Done
        Percent Complete30% 100%
        Open/ClosedOpen Closed
    2010-07-13 mpetazzoni Should be Finished on2010-02-03 2010-08-06
        Priority6 7 - High
        StatusNone In Progress
        Percent Complete0% 30%
        Assigned toNone mpetazzoni

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code