helpSavannah Administration - Support: sr #109942, restore automake-commit mail?

 
 

sr #109942: restore automake-commit mail?

Submitted by:  Karl Berry <karl>
Submitted on:  Tue 27 Aug 2019 01:44:25 PM UTC  
 
Category:  Project mailing lists Priority:  5 - Normal
Severity:  3 - Normal Status:  None
Assigned to:  Ineiev <ineiev> Operating System:  None
Open/Closed:  Closed

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Fri 10 Jan 2020 06:29:16 AM UTC, comment #24: 

Okay.  Have added a commitBrowseURL config and it should show up in the next commit messages.

Bob Proulx <rwp>
Site Administrator
Thu 09 Jan 2020 10:19:12 PM UTC, comment #23: 

including urls sounds desirable, because, why not.

i have no argument or desire for any other change. thanks.

Karl Berry <karl>
Site Administrator
Thu 09 Jan 2020 07:58:54 PM UTC, comment #22: 

> *-commits -- the commit with full new versions of modified files.


> *-diffs -- the commit with diffs against the previous version.


When I look at the old emacs-commit list archive I don't see a full new version of modified files posted there.  What I see is a URL to the viewvc web archive of the commit in a message that is very similar to the git multimail introduction header message for a series of commits.  So even the older CVS mail hook did not seem to send out full copies of modified files.

git-multimail has a commitBrowseURL variable and feature that allows adding a URL to the web archive of commits.  It was not in the earlier versions but is available in later versions.  This could be added to any of the git-multimail configurations and then a URL would appear for each of the messages.

git-multimail allows sending the introduction message to a different mailing list than the individual commit messages.  The various variables are "mailinglist" and "commitlist" and "refchangelist" and perhaps some others.  If only mailinglist is set then both messages go there.  If both are set then each is as specified.  Or it may be the other way around.  Some experimentation would be needed.  But I am sure it is possible to send a message similar to the old emacs-commit messages in addition to the current emacs-diffs messages.

Bob Proulx <rwp>
Site Administrator
Thu 09 Jan 2020 07:44:40 PM UTC, comment #21: 

> Indeed, my next commit to automake came in as karl@gnu.org. While not exactly correct, it seems good enough.


That comes from your registered email in the database.  If that were changed then the outgoing mail would also be changed.

Outgoing mail from vcs0 would, if nothing were configured, come from $USER@vcs0.savannah.gnu.org and that's not good.  Have configured email masquerading so that all mail comes from $USER@savannah.gnu.org hiding the actual host name.  But email from that domain isn't valid so obviously also is not good.

Therefore mail from individuals (commits for git, svn, cvs, et al are from the user via ssh login) is mapped through a mapping database that maps usernames to the registered email address.  That is how it arrived at your -email is unavailable- address above.  It mapped it to the registered address in the database.

The git-multimail program we have been using does not extract the git author email address nor the git committer email address from the git commit changeset.  I am sure it could.  I suppose the author email would be more appropriate than the committer address.  If someone wanted to code up the python for it then that would be fine with me.  I am not a python person and the inner workings of git multimail are obscure to me.

I am not voting for or against anything here.  I am simply reporting the facts of the situation as they sit right now.

Bob Proulx <rwp>
Site Administrator
Tue 07 Jan 2020 10:23:29 PM UTC, comment #20: 

Indeed, my next commit to automake came in as karl@gnu.org. While not exactly correct, it seems good enough. I imagine it ends up as <savannah-user-name>@gnu.org, which may or may not be a valid email address. But that's ok.

*-commits -- the commit with full new versions of modified files.
*-diffs -- the commit with diffs against the previous version.

As I wrote, in the non-git world (log_accum), these were two separate things. The Savannah project admin UI was, for a simple example: https://savannah.gnu.org/cvs/admin/?group=fontutils
which just sends the full commit mail, no diffs.

And it supported other features, filtering by directory, etc. See ...=emacs for a more complex example, no longer active at all afaik. I think none of this is supported with git commit email.

Anyway, since the Emacs people haven't worried about it for 12 years, there's no point in us worrying about it now.

I am, happily, finally closing this bug. Thanks again!

Karl Berry <karl>
Site Administrator
Tue 07 Jan 2020 04:55:37 AM UTC, comment #19: 

I am pretty sure that the mail being from root even though I ran it with sudo to be "karl" was because it probably pulled root from USER or one of the other variables. I did not expect that and after it happened I didn't want to run it twice. Seems like it probably should work when done for real. Since then it is a login as the actual user.

I am okay with leaving the from address from the committer. I had forgotten that passing through the mailing lists causes the fixup as it goes through the Exim config. So it will be just like all of the mailing lists. That's good to be all the same.

I looked at the emacs config and the file has not changed since 2014.

[multimailhook]
        mailinglist = -email is unavailable-
        emailPrefix = ""

-rw-rw-r-- 1 root emacs 229 Nov 18  2014 config

That seems pretty simple and explicit.  But I don't know.  I can't remember what I had for breakfast yesterday since I have slept since then.

What is the difference between "commits" and "diffs"?  That distinction has no meaning to me.  Aren't they both the same?  Not understanding that is hampering my understanding of the problem.

We could ask Glenn Morris about the thinking at the time as I am pretty sure he was involved in it.  (But I have little memory of the emacs transition. It is even possible that I was involved and have no memory of it.)

There are 265 post-receive scripts in place that are not executable. But that is because at least the ones I spot checked are all empty comments-only templates. So not sure I would want to simply enable all of them.

When I grep to list non-executable post-receive hooks that are using git-multimail I find only the single test-project which I used for testing but then disabled afterward. All of the others using git-multimail are executable.

I did not look at all of the other non-git-multimail hooks. I don't know how many of those there are. But there are at least a few that do things different. Those are all uniquely different.

Bob Proulx <rwp>
Site Administrator
Mon 06 Jan 2020 10:16:05 PM UTC, comment #18: 

1) the mail came through, yay!! thank you so much. sorry i left it up to you guys.

2) as you noted, it came From: root@freefriends.org. Ok, we'll find out about the From: address next time we commit something.

3) I strongly think it should be the real committer address, not the list. The solution to DMARC, imho, is dmarc_moderation_action=munge from, which is already set for automake-commit. (and i believe pretty much all lists have either had that set or the headers/footers removed, etc., per Ian.)

4) are you sure emacs-commit was intended to be "replaced"? it feels unexpected to me. in the non-git world, emacs-commit was the full commits and emacs-diffs was the diffs. there was the savannah web ui for project admins to set them independently.

barring evidence to the contrary, i suspect the full commit email simply stopped working when they switched to git and nobody noticed/reported.

4) maybe check that that all post-receive files are x?

thanks again!!

Karl Berry <karl>
Site Administrator
Mon 06 Jan 2020 09:49:45 AM UTC, comment #17: 

New question/problem: Given the state of sites with strict DMARC policies, should we set the From: address to the mailing list?

Bob Proulx <rwp>
Site Administrator
Mon 06 Jan 2020 09:13:52 AM UTC, comment #16: 

Among other things the post-receive hook file was not executable.  Therefore it was not being invoked.  I chmod a+x post-receive the file so that it would be executable.

I simulated the most recent commit.  The mail log says it has sent email out to automake-commit.  So perhaps it is working now.

The previous commit (HEAD^) was cf27a3dfc64b877524bea0505fe2820d6bfe6251 and the current commit (HEAD) is b87f297426c2d79494b707d0fecddaff2b6eaefd therefore to push out that set of commits (there was only one in that set) one must be in the git directory, pipe the commit hash values into the hook script as stdin, and run the hook as the user that did the commit.

root@vcs0:/net/vcs/git/automake.git# echo cf27a3dfc64b877524bea0505fe2820d6bfe6251 b87f297426c2d79494b707d0fecddaff2b6eaefd refs/heads/master | sudo -u karl ./hooks/post-receive
Sending notification emails to: -email is unavailable-

Which seems like it should have worked perfectly.  But in this case the actual email has a From: address of root when it should not have. Hmm...  Oh well.  I think in a real commit case that it will have the correct address though.  I checked and the mail was sent to the mailing list and appears in the raw mailbox archives.  I did not wait for the */30 cronjob to thread the message.  I'll check that tomorrow.

I also noticed a bug that has been in the git-multimail.py program for a while.  The revision header message complains that "reply_to" is not set (it is for the refchange message though) and does not include the Mail-Followup-To header on that message.  It is included in the individual refchange messages.  The python code was too inscrutable for me to discern the problem though.

Bob Proulx <rwp>
Site Administrator
Mon 06 Jan 2020 06:49:19 AM UTC, comment #15: 

https://lists.gnu.org/archive/html/emacs-commit/ was apparently replaced by https://lists.gnu.org/archive/html/emacs-diffs/ which is current to the present time.

Bob Proulx <rwp>
Site Administrator
Mon 06 Jan 2020 06:30:57 AM UTC, comment #14: 

Strike that last comment.  I spoke too soon.  I had made a silly mistake in my instrumented script.  stdin is coming through with the commit has ids okay.  Sorry for the noise.

I am digging into it.  That's all I can say.

Bob Proulx <rwp>
Site Administrator
Mon 06 Jan 2020 06:27:47 AM UTC, comment #13: 

Something very strange is going on because I instrumented a hook script in a test repository and while the post-receive hook is supposed to get information on stdin my instrumented hook script is not receiving anything.  That's probably at the root cause of why notifications are not going out.  Something is not triggering the hook scripts as expected.  They are getting run however.

Bob Proulx <rwp>
Site Administrator
Sun 05 Jan 2020 02:03:19 AM UTC, comment #12: 

Thanks for trying. Unfortunately it did not help. I just pushed another commit to automake.git more than an hour ago, and no mail has shown up on automake-commit, and mail archives have not been updated. It remains stuck in 2017.

Karl Berry <karl>
Site Administrator
Tue 31 Dec 2019 06:55:28 AM UTC, comment #11: 

I've just copied the hooks from guix.git (it looks like they work there); let us see if it helps.

Ineiev <ineiev>
Site AdministratorIn charge of this item.
Sat 21 Dec 2019 01:23:05 AM UTC, comment #10: 

Just to mention - the situation hasn't changed :(. Jim just pushed another change in automake.git:
http://git.savannah.gnu.org/cgit/automake.git/log/

And nothing was sent to automake-commit:
https://lists.gnu.org/archive/html/automake-commit/

Karl Berry <karl>
Site Administrator
Mon 21 Oct 2019 01:15:11 AM UTC, comment #9: 

More data points (sorry for not being able to really fix it): emacs-commit is also stuck in 2008:
https://lists.gnu.org/archive/html/emacs-commit/

whereas emacs-diffs is current:
https://lists.gnu.org/archive/html/emacs-diffs/

and of course there are lots of commits to emacs.git:
http://git.savannah.gnu.org/cgit/emacs.git/log/

I do see some commit lists that are up to date, though, e.g.:
https://lists.gnu.org/archive/html/guix-commits/
https://lists.gnu.org/archive/html/auctex-commit/
https://lists.gnu.org/archive/html/guile-commits/

Running ls -lt commit/2019-10 in lists:/home/mharc/mbox shows 17 such lists getting mail this month. Doesn't seem like very many.

I also looked in lists:/spam/automake-commit, but no real mail was getting caught there, only spam (as one would hope).

I don't know what to make of it all. --thanks, karl.

Karl Berry <karl>
Site Administrator
Fri 18 Oct 2019 09:51:24 PM UTC, comment #8: 

I believe the lack of commit mail is not specific to automake. I see recent commits to texinfo git (http://git.savannah.gnu.org/cgit/texinfo.git/log/), but the texinfo-commits list stops in 2018 (https://lists.gnu.org/archive/html/texinfo-commits/).

Karl Berry <karl>
Site Administrator
Wed 16 Oct 2019 06:05:54 AM UTC, comment #7: 

I'm not that technical that I can help you out.

ofs is offset, I just copied and pasted the page link
where you can see the first commit after 1-1-2017 from
Mathieu Lirzin who is a member since 23-02-2017, maybe something
has changed at that point. (Could it be a key or something?)
Could the list accidentaly be blocked?
Dont know If you may need to write to:
https://lists.gnu.org/mailman/listinfo/mailman/

-----------
https://www.fsf.org/resources/webmail-systems/
-----------

bendikker <bendukker>
Tue 15 Oct 2019 09:23:35 PM UTC, comment #6: 

I don't believe it is necessary for a list to be attached to be a group to receive commit mail. That wouldn't make sense. But anyway, I just attached it (following the instructions at ListServer on the wiki), since it may as well be there.

Your other note - I don't know what ofs=100 means, but http://git.savannah.gnu.org/cgit/automake.git/log/ (without that parameter) shows recent commits, as does http://git.savannah.gnu.org/cgit/automake.git/commit/. And if you actually check out the repository you will see them too.

Thanks. -k

Karl Berry <karl>
Site Administrator
Tue 15 Oct 2019 08:03:15 PM UTC, comment #5: 

Was just curious, am a noob.

automake-commit is not on the list
https://savannah.gnu.org/mail/?group=automake

There it stops:
Mathieu Lirzin member since 2-2017
http://git.savannah.gnu.org/cgit/automake.git/log/?ofs=100

bendikker <bendukker>
Tue 15 Oct 2019 03:51:34 PM UTC, comment #4: 

unfortunately it did not work. Jim made a couple more commits to automake in the last day or two, and nothing showed up on automake-commit. still stuck in 2017.
https://lists.gnu.org/archive/html/automake-commit/

Karl Berry <karl>
Site Administrator
Wed 18 Sep 2019 07:31:33 AM UTC, comment #3: 

I am not exactly sure what is going on with this hook and why it is not functioning.  It looks like it should be working.  But I do agree that there is nothing in the archive since 2017 and therefore somewhere it is not working.  The configuration as set appears correct all around.

Because the current is not working, and an updated version of git_multimail was available, I shifted to using the newer version.  An example of it is the Emacs list commit hook.  Just because trying it seemed better than not trying it.

I can simulate a push and cause it to generate an email but I have no more time tonight to try it.  Later...

As far as the timestamps go...  Git does not save a history of changes and does not keep track of when the branch was sync'd.  The timestamps shown are the timestamps that were in the branch before in Jim's repository and then pushed.

Bob Proulx <rwp>
Site Administrator
Mon 16 Sep 2019 03:04:26 PM UTC, comment #2: 

unfortunately, no luck. Jim Meyering pushed a commit to automake a couple days ago:
http://git.savannah.gnu.org/cgit/automake.git/commit/?id=309a6c477eec80b847078699303c65ccd7787eb0

(Although I cannot see the correct date in the git interface, he said he pushed it on September 15. From earlier mail from him, I know that the Sep 02 date shown there is when he committed it locally. I don't know how to see the real commit date.)

Anyway, there is still nothing in the automake-commit archives since 1 Jan 2017:
https://lists.gnu.org/archive/html/automake-commit/

I also didn't receive any commit mail, though I'm on the list.

Help again?

Thanks,
Karl

Karl Berry <karl>
Site Administrator
Fri 30 Aug 2019 08:08:15 AM UTC, comment #1: 

I used the instructions for enabling notifications from Git.mdwn, but they didn't apply directly; let us see the result.

Ineiev <ineiev>
Site AdministratorIn charge of this item.
Tue 27 Aug 2019 01:44:25 PM UTC, original submission:  

automake-commit stopped getting commit mail in 2017 (https://lists.gnu.org/archive/html/automake-commit). Ineiev, Bob, could this be restored, please? Sorry, but I don't know how to do it. Thanks.

Karl Berry <karl>
Site Administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rwp (Posted a comment)
  • -email is unavailable- added by ineiev (Posted a comment)
  • -email is unavailable- added by karl (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

     

     

    Follow 4 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-01-07 karl Open/ClosedOpen => Closed
    2019-10-15 karl StatusReady For Test => None
    2019-08-30 ineiev StatusNone => Ready For Test
        Assigned toNone => ineiev

    Back to the top


    Powered by Savane 3.5