taskMapOSMatic - Tasks: task #10148, Switch to osmosis for DB updates

 
 

task #10148: Switch to osmosis for DB updates

Submitted by:  David Decotigny <daviddecotigny>
Submitted on:  Wed 03 Feb 2010 12:59:33 PM UTC  
 
Should Start On:  Wed 03 Feb 2010 12:00:00 AM UTC Should be Finished on:  Fri 06 Aug 2010 10:00:00 PM UTC
Category:  database Priority:  7 - High
Status:  Done Privacy:  Public
Percent Complete:  100% Assigned to:  Maxime Petazzoni <mpetazzoni>
Open/Closed:  Closed Release:  None
Effort:  0.00

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Tue 02 Nov 2010 08:56:18 AM UTC, comment #3: 

Waouh, I didn't know this. Great news !

David Decotigny <daviddecotigny>
Project Administrator
Fri 29 Oct 2010 09:08:58 PM UTC, comment #2: 

This is done. The GIS database now hosted on a separate server uses 15-min update cycles with Osmosis.

Maxime Petazzoni <mpetazzoni>
Project AdministratorIn charge of this item.
Tue 13 Jul 2010 12:37:48 PM UTC, comment #1: 

The new server will use Osmosis for minutely updates. Migration is in progress, but hardware issues on the VM's host are making things difficult for now.

Maxime Petazzoni <mpetazzoni>
Project AdministratorIn charge of this item.
Wed 03 Feb 2010 12:59:33 PM UTC, original submission:  

From Thomas:
«
Les diff hourly et minute ont été supprimés, mais ils ont été
visiblement remplacés par une nouvelle méthode de synchronisation,
basée sur Osmosis:

 http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage#Replication_Tasks

Les données de différences sont apparemment dispo sur:
 http://planet.openstreetmap.org/hour-replicate/
 http://planet.openstreetmap.org/minute-replicate/

D'après ce que je comprends de
http://lists.openstreetmap.org/pipermail/osmosis-dev/2009-November/000310.html,
ce nouveau système de diff a l'air d'utiliser directement des
transactions PostgreSQL. Je suspecte que ce soit potentiellement plus
efficace que la méthode actuelle. À étudier, peut-être demander sur la
liste si il y a des compétences dans le domaine.

A+
»

Looks like osmosis is promising. And, in the short term, we will HAVE to use it.

Never looked deep enough into it, but it reveals an interesting pipeline-based architecture, even though osm2pgsql still seems to be necesary to update the DB (but maybe I'm missing something).

Anyway, this could be of interest for us, too:
http://lists.openstreetmap.org/pipermail/osmosis-dev/2009-November/000313.html
Especially this script: https://svn.toolserver.org/svnroot/mazder/diff-import/load-next

David Decotigny <daviddecotigny>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mpetazzoni (Posted a comment)
  • -email is unavailable- added by daviddecotigny (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2010-10-29 mpetazzoni StatusIn Progress => Done
        Percent Complete30% => 100%
        Open/ClosedOpen => Closed
    2010-07-13 mpetazzoni Should be Finished on2010-02-03 => 2010-08-06
        Priority6 => 7 - High
        StatusNone => In Progress
        Percent Complete0% => 30%
        Assigned toNone => mpetazzoni

    Back to the top


    Powered by Savane 3.7