bugMapOSMatic - Bugs: bug #36282, Non recovered crash of the...

 
 

bug #36282: Non recovered crash of the rendering pipeline

Submitted by:  David Mentré <dmentre>
Submitted on:  Tue 24 Apr 2012 11:43:33 AM UTC  
 
Category: ocitysmapSeverity: 4 - Important
Priority: 7 - HighStatus: Confirmed
Privacy: PublicAssigned to: Maxime Petazzoni <mpetazzoni>
Open/Closed: OpenRelease: 2012.04

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Tue 24 Apr 2012 02:43:52 PM UTC, comment #3:

This might be of interest:

http://djangosnippets.org/snippets/1707/

Maxime Petazzoni <mpetazzoni>
Project AdministratorIn charge of this item.
Tue 24 Apr 2012 12:25:26 PM UTC, comment #2:

Thomas's analysis: a GEOS Intersection() error in the son rendering process triggers a crash in the father rendering daemon because the father is losing its connection to the database.

David Mentré <dmentre>
Project Administrator
Tue 24 Apr 2012 11:49:28 AM UTC, comment #1:

Backtrace of the father process:

"""
g:prod-instance maposmatic-prod/maposmatic> maposmatic> ./scripts/wrapper.py ./scripts/daemon.py
Traceback (most recent call last):
File "/home/maposmatic/maposmatic-prod/maposmatic/./scripts/daemon.py", line 253, in <module>
daemon.serve()
File "/home/maposmatic/maposmatic-prod/maposmatic/./scripts/daemon.py", line 78, in serve
self.dispatch(job)
File "/home/maposmatic/maposmatic-prod/maposmatic/./scripts/daemon.py", line 98, in dispatch
return self.render(job, 'maposmaticd_%d_' % os.getpid())
File "/home/maposmatic/maposmatic-prod/maposmatic/./scripts/daemon.py", line 113, in render
job.end_rendering(_RESULT_MSGS[ret])
File "/home/maposmatic/maposmatic-prod/maposmatic/www/maposmatic/models.py", line 140, in end_rendering
self.save()
File "/usr/lib/pymodules/python2.6/django/db/models/base.py", line 435, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/lib/pymodules/python2.6/django/db/models/base.py", line 497, in save_base
manager.using(using).filter(pk=pk_val).exists())):
File "/usr/lib/pymodules/python2.6/django/db/models/query.py", line 496, in exists
return self.query.has_results(using=self.db)
File "/usr/lib/pymodules/python2.6/django/db/models/sql/query.py", line 417, in has_results
return bool(compiler.execute_sql(SINGLE))
File "/usr/lib/pymodules/python2.6/django/db/models/sql/compiler.py", line 727, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/pymodules/python2.6/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
django.db.utils.DatabaseError: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.

zsh: exit 1 ./scripts/wrapper.py ./scripts/daemon.py
"""

David Mentré <dmentre>
Project Administrator
Tue 24 Apr 2012 11:43:33 AM UTC, original submission:

Sometimes, the rendering process crashes, crashing at the same time the daemon that handles jobs. The whole rendering pipeline is thus blocked.

The daemon should handle gracefully such crash of the rendering process, but the logic is apparently failing sometimes.

Here is an example of job that blocked everything:
"""
An error occured while rendering job #45269!

Traceback (most recent call last):
File "/home/maposmatic/maposmatic-prod/maposmatic/scripts/render.py", line 334, in run
output_formats, prefix)
File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/__init__.py", line 465, in render
file_prefix)
File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/__init__.py", line 520, in _render_one
renderer = renderer_cls(self._db, config, tmpdir, dpi, file_prefix)
File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/layoutlib/single_page_renderers.py", line 548, in _init_
SinglePageRenderer.__init__(self, db, rc, tmpdir, dpi, file_prefix, None)
File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/layoutlib/single_page_renderers.py", line 78, in _init_
rc.i18n)
File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/indexlib/../indexlib/indexer.py", line 67, in _init_
+ self._list_villages(db, polygon_wkt))
File "/home/maposmatic/maposmatic-prod/ocitysmap/ocitysmap2/indexlib/../indexlib/indexer.py", line 343, in _list_amenities
cursor.execute(query)
InternalError: GEOS Intersection() threw an error!

Job information:

administrative_city: Cazenovia, Madison County, New York, United States of America
administrative_osmid: -175948
endofrendering_time: None
id: 45269
index_queue_at_submission: 0
lat_bottom_right: None
lat_upper_left: None
layout: plain
lon_bottom_right: None
lon_upper_left: None
map_language: en_US.UTF-8
maptitle: Cazenovia, New York
nonce: XXXXXXXXXXXXXXX
paper_height_mm: 594
paper_width_mm: 420
resultmsg: None
startofrendering_time: 2012-04-24 13:33:25.548864
status: 1
stylesheet: Default
submission_time: 2012-04-24 04:41:10.100107
submitterip: XXXXXXXXXXXXXXX

You can view the job page at <http://www.maposmatic.org/jobs/45269>.
"""

David Mentré <dmentre>
Project Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mpetazzoni (Posted a comment)
  • -unavailable- added by dmentre (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 4 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Tue 24 Apr 2012 06:40:02 PM UTCmpetazzoniSeverity3 - Normal=>4 - Important
      Priority5 - Normal=>7 - High
      Assigned toNone=>mpetazzoni
      ReleaseNone=>2012.04

    Back to the top


    Powered by Savane 3.1-cleanup