taskSavannah Administration - Tasks: task #15599, Add bounce processing to Savannah

 
 

task #15599: Add bounce processing to Savannah

Submitted by:  Ian Kelling <iank>
Submitted on:  Sat 18 Apr 2020 11:48:49 PM UTC  
 
Should Start On:  Sat 18 Apr 2020 12:00:00 AM UTC Should be Finished on:  Sat 18 Apr 2020 12:00:00 AM UTC
Category:  System Priority:  7 - High
Status:  None Privacy:  Public
Percent Complete:  0% Assigned to:  None
Open/Closed:  Open Effort:  0.00

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Sat 18 Apr 2020 11:48:49 PM UTC, original submission:  

frontend1.savannah.gnu.org is sending out email notifications through
eggs from www-data@savannah.gnu.org. Sometimes they bounce, and
currently we are just ignoring those bounces. A lot of those bounce
messages indicate that we should not be sending any more email to that
address, either its no longer a valid email or a few other reasons, and
continuing to do so hurts the sender reputation of eggs. We've been
having significant trouble with our server getting blacklisted over the
last year or so, so I'm giving this item a high priority.

I could also make a separate email server for Savannah, but it could
hurt the reputation of our ip range, or the parent domain, gnu.org, we
don't know exactly, email hosters are typically running some secret
system.

So, I'm hoping someone at savannah will Savannah implement some code to
start processing these bounces and disabling sending notifications, or
email entirely to that address if the bounce calls for it.

I would make eggs deliver the bounces to a savannah machine, then the
Savannah bounce processor would maybe run from a cronjob or directly
from the mta, look at the bounce and decide if the email address in the
bounce should be disabled then discard the bounce email. Ideally, the
Savannah web ui would have some way to indicate that the email is
disabled, so a user can reenable it in case they update their email
server/hosting so it becomes a valid email. In discourse, they call the
whole account disabled.

Many web applications that send notifications seem to have created their
own system for bounce processing. A few examples: GNU mailman disables
any email that bounces too many times within a certain period. Civicrm
searches for the reason within the body of the bounce, and tries to
classify it into categories, then has a separate threshold of # of
bounces within each category. Discourse does something
similar,https://meta.discourse.org/t/handling-bouncing-e-mails/45343. I've
found it was more reliable to alter the Civicrm way into a blacklist of
strings and ignore the rest. Anyways, the GNU Mailman way looks to be
the simplest to implement.

Some headers from an example message that bounceed:

X-Savane-Tracker: bugs
Subject: [bug #redacted] redacted

In my research, I found an advanced bounce parsing library in ruby or
perl, https://github.com/sisimai, but I'm guessing it would be pretty
easy to use any popular email parsing library in a high level language
to build your own totally sufficient bounce processing with a few hours
or days of effort. Also, I've found the GNU Mailman code to be fairly
readable, so it might be a good example.

Ian Kelling <iank>
Site Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by iank (Submitted the item)
  • -email is unavailable- added by iank
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follows 1 latest change.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-04-18 iank Carbon-Copy- => Added iank@fsf.org

    Back to the top


    Powered by Savane 3.5