bugMapOSMatic - Bugs: bug #36282, Non recovered crash of the...

 
 

bug #36282: Non recovered crash of the rendering pipeline

Submitter:  David Mentré <dmentre>
Submitted:  Tue 24 Apr 2012 11:43:33 AM UTC
   
 
Category:  ocitysmap Severity:  4 - Important
Priority:  * 7 - High Status:  Confirmed
Privacy:  Public Assigned to:  mpetazzoni
Open/Closed:  Open Release:  2012.04
* Mandatory Fields

Add a New Comment Rich Markup
   

Tue 24 Apr 2012 02:43:52 PM UTC, comment #3: 

This might be of interest:

http://djangosnippets.org/snippets/1707/

Maxime Petazzoni <mpetazzoni>
Group administrator
Tue 24 Apr 2012 12:25:26 PM UTC, comment #2: 

Thomas's analysis: a GEOS Intersection() error in the son rendering process triggers a crash in the father rendering daemon because the father is losing its connection to the database.

David Mentré <dmentre>
Group administrator
Tue 24 Apr 2012 11:49:28 AM UTC, comment #1: 

Backtrace of the father process:

"""
g:prod-instance maposmatic-prod/maposmatic> maposmatic> ./scripts/wrapper.py ./scripts/daemon.py
Traceback (most recent call last):
  File "/home/maposmatic/maposmatic-prod/maposmatic/./scripts/daemon.py", line 253, in <module>
    daemon.serve()
  File "/home/maposmatic/maposmatic-prod/maposmatic/./scripts/daemon.py", line 78, in serve
    self.dispatch(job)
  File "/home/maposmatic/maposmatic-prod/maposmatic/./scripts/daemon.py", line 98, in dispatch
    return self.render(job, 'maposmaticd_%d_' % os.getpid())
  File "/home/maposmatic/maposmatic-prod/maposmatic/./scripts/daemon.py", line 113, in render
    job.end_rendering(_RESULT_MSGS[ret])
  File "/home/maposmatic/maposmatic-prod/maposmatic/www/maposmatic/models.py", line 140, in end_rendering
    self.save()
  File "/usr/lib/pymodules/python2.6/django/db/models/base.py", line 435, in save
    self.save_base(using=using, force_insert=force_insert, force_update=force_update)
  File "/usr/lib/pymodules/python2.6/django/db/models/base.py", line 497, in save_base
    manager.using(using).filter(pk=pk_val).exists())):
  File "/usr/lib/pymodules/python2.6/django/db/models/query.py", line 496, in exists
    return self.query.has_results(using=self.db)
  File "/usr/lib/pymodules/python2.6/django/db/models/sql/query.py", line 417, in has_results
    return bool(compiler.execute_sql(SINGLE))
  File "/usr/lib/pymodules/python2.6/django/db/models/sql/compiler.py", line 727, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/pymodules/python2.6/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
    return self.cursor.execute(query, args)
django.db.utils.DatabaseError: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.

zsh: exit 1     ./scripts/wrapper.py ./scripts/daemon.py
"""

David Mentré <dmentre>
Group administrator
Tue 24 Apr 2012 11:43:33 AM UTC, original submission:  

Sometimes, the rendering process crashes, crashing at the same time the daemon that handles jobs. The whole rendering pipeline is thus blocked.

The daemon should handle gracefully such crash of the rendering process, but the logic is apparently failing sometimes.

Here is an example of job that blocked everything:
"""
An error occured while rendering job #45269!

Traceback (most recent call last):
 File "/home/maposmatic/maposmatic-prod/maposmatic/scripts/render.py", line 334, in run
   output_formats, prefix)
 File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/__init__.py", line 465, in render
   file_prefix)
 File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/__init__.py", line 520, in _render_one
   renderer = renderer_cls(self._db, config, tmpdir, dpi, file_prefix)
 File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/layoutlib/single_page_renderers.py", line 548, in _init_
   SinglePageRenderer.__init__(self, db, rc, tmpdir, dpi, file_prefix, None)
 File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/layoutlib/single_page_renderers.py", line 78, in _init_
   rc.i18n)
 File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/indexlib/../indexlib/indexer.py", line 67, in _init_
   + self._list_villages(db, polygon_wkt))
 File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/indexlib/../indexlib/indexer.py", line 343, in _list_amenities
   cursor.execute(query)
InternalError: GEOS Intersection() threw an error!



Job information:

 administrative_city: Cazenovia, Madison County, New York, United States of America
 administrative_osmid: -175948
 endofrendering_time: None
 id: 45269
 index_queue_at_submission: 0
 lat_bottom_right: None
 lat_upper_left: None
 layout: plain
 lon_bottom_right: None
 lon_upper_left: None
 map_language: en_US.UTF-8
 maptitle: Cazenovia, New York
 nonce: XXXXXXXXXXXXXXX
 paper_height_mm: 594
 paper_width_mm: 420
 resultmsg: None
 startofrendering_time: 2012-04-24 13:33:25.548864
 status: 1
 stylesheet: Default
 submission_time: 2012-04-24 04:41:10.100107
 submitterip: XXXXXXXXXXXXXXX

You can view the job page at <http://www.maposmatic.org/jobs/45269>.
"""

David Mentré <dmentre>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mpetazzoni (Posted a comment)
  • -email is unavailable- added by dmentre (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2012-04-24 mpetazzoni Severity3 - Normal 4 - Important
        Priority5 - Normal 7 - High
        Assigned toNone mpetazzoni
        ReleaseNone 2012.04

    Back to the top

    Powered by Savane 3.13-3230.
    Corresponding source code