taskgNewSense GNU/Linux - Tasks: task #12794, List differences between Debian...

 
 

You are not allowed to post comments on this tracker with your current authentication level.

task #12794: List differences between Debian and gNewSense

Submitter:  Sam Geeraerts <samgee>
Submitted:  Sun 01 Sep 2013 08:49:37 AM UTC
   
 
Should Start On:  Sat 31 Aug 2013 10:00:00 PM UTC Should be Finished on:  Sat 31 Aug 2013 10:00:00 PM UTC
Category:  Documentation Priority:  5 - Normal
Status:  In Progress Privacy:  Public
Assigned to:  rsd Open/Closed:  Open

Jump to the original submission

Thu 02 Mar 2017 12:57:47 AM UTC, comment #23: 

@samgee,

I've re-written the script in python3:

  https://pypi.python.org/pypi/gns-deb-diff

I would like to test the script on gnewsense.org

Would it be possible to enable xmlrpc for gNewSense's wiki?

rsiddharth <rsd>
Group Member
Sun 12 Jan 2014 05:34:54 AM UTC, comment #22: 


> It would be great if the automated update will only happen if 
> the page actually needs to be changed. Then we could run it in
> a cron job.


Thinking loudly (in the light of recent spam attacks):

I figure, we got to do something to prevent spammers abusing the site with XML-RPC, here is one solution I thought of:

  • Maintain two copies of wikiconfig.py -- one with XML-RPC disabled & the other with XML-RPC enabled; the former is put to use & the latter kept somewhere else in the server.


And we can cook up a script to switch between the two wikiconfig.py files. Let us called this `xml-rpc-switch` script.

  • If the cron job (to run the gns-deb-diff script) is run in the same server that hosts the wiki, then:


** The `xml-rpc-switch` script is run first to effectively enable XML-PRC (by switching to another wikiconfig.py)

** `gns-deb-diff` script is run.

** The `xml-rpc-switch` script is run to switch back to the old wikiconfig.py; effectively disabling XML-RPC again.

This way, we can enable XML-RPC only when the gns-deb-diff script is run. What do you think?


rsiddharth <rsd>
Group Member
Fri 03 Jan 2014 03:49:32 AM UTC, comment #21: 


>  Another option is setting up HTTP authentication for
>  action=xmlrpc2 requests. I'll gladly take any Apache
>  configuration hints you may have to that end.


I looked here & here, still clueless. Perhaps, you may find something there that might be useful.

>  There have been some more package updates...If you're up for
>  it, do a live run of the script to see if it'll be friends
>  with our wiki.


I ran the script, it updated the wiki.

At the moment, the list of parkes packages (for which the script searches for the README.gNewSense file) is maintained in the git repo with the gns-deb-diff script. Can we move this file to the gNewSense's bzr repo? Perhaps at:


  bzr.savannah.gnu.org/lh/gnewsense/packages-parkes-list/package.list


or some other appropriate location, so that whenever a package is added to `gnewsense/packages-parkes/`, the name of the package is added to this file & the script can be accordingly modified to use this packages.list file at the gNewSense bzr repo.

Thanks.


rsiddharth <rsd>
Group Member
Thu 02 Jan 2014 07:56:26 PM UTC, comment #20: 

The xmlrpc frontend uses the same authorization process as the Web frontend, minus the textcha. I hoped that MoinMoin's plugin system could be of help, but that only allows creating new methods, not overriding xmlrpc to implement different authorization. So separate authorization would have to be implemented in MoinMoin itself. I suspect that's a bit of work.

Another option is setting up HTTP authentication for action=xmlrpc2 requests. I'll gladly take any Apache configuration hints you may have to that end. No sign of abuse so far, though.

There have been some more package updates. I've not tested your updated script, but the code makes sense. If you're up for it, do a live run of the script to see if it'll be friends with our wiki.

Sam Geeraerts <samgee>
Group administrator
Thu 02 Jan 2014 11:51:42 AM UTC, comment #19: 

Now, I'm really alarmed at the prospect of spammers being able to do massive editing using xmlrpc.

Is it possible to enable xmlrpc for just one user in the wiki?

If that's possible we can create a user called "xmlprcuser" and use it to update the wiki.

> It would be great if the automated update will only happen if
> the page actually needs to be changed. Then we could run it in
> a cron job.


I've updated the script. It remotely updates the wiki only when the newly generated wiki page differs from the old wiki page.

rsiddharth <rsd>
Group Member
Wed 01 Jan 2014 07:40:05 PM UTC, comment #18: 

I had a closer look at xmlrpc again. I see that Debian have it disabled "to mitigate unspecified security issues" [1]. It's been disabled by default in MoinMoin since 1.9.1 for security reasons [2]. I didn't find any concrete reports of security issues. It at least lowers the barrier for spam attacks. It might also give some wiki editors adventurous ideas about massive editing.

I've enabled xmlrpc for now. Don't tell the spammers (though I hope the EditorGroup will remain a sufficient measure if they do find out). Use this feature sparingly and cautiously. And please help me look out for any suspicious activity on the wiki.

It would be great if the automated update will only happen if the page actually needs to be changed. Then we could run it in a cron job.

[1] http://anonscm.debian.org/gitweb/?p=collab-maint/wiki.debian.org.git;a=blob;f=etc/moin/wikiconfig.py;hb=HEAD
[2] http://moinmo.in/SecurityFixes

Sam Geeraerts <samgee>
Group administrator
Sun 29 Dec 2013 03:37:59 PM UTC, comment #17: 

If there is no security issues with XML-RPC, can we use it to update the wiki page?

Thanks.

rsiddharth <rsd>
Group Member
Sat 28 Dec 2013 10:30:01 PM UTC, comment #16: 

I remember I looked at XML-RPC for a bit as a potential aid in our wiki spam problem. I was concerned about security, but I don't remember if that was based on facts or just a feeling. I would have to delve into it again.

A MoinMacro would have to get it's data from Savannah's Loggerhead (Bazaar web interface). It can be very slow at times. I think this would make the page very slow to load.

Sam Geeraerts <samgee>
Group administrator
Fri 27 Dec 2013 02:56:51 PM UTC, comment #15: 

Thanks a ton samgee, for adding the relevant README.gNewSense files to the remaining packages.

My next task, will be to figure out a way to automatically update the wiki. With regard to this, sirgazil pointed me to MoinMacros as a possible solution & I was wondering the about the possibility of using the MoinAPI.

What do you think might be the best solution?

Thanks.

rsiddharth <rsd>
Group Member
Wed 25 Dec 2013 09:14:45 PM UTC, comment #14: 

I've updated the remaining packages, clarified a few others and added some which I apparently forgot to push to Savannah.

Sam Geeraerts <samgee>
Group administrator
Mon 23 Dec 2013 12:24:13 PM UTC, comment #13: 

Sorry for the hastily written documentation. I've updated the gns-deb-diff-notes wiki page. I guess the description should be clearer now.

I've finished updating the existing README.gNewSense files with the `Changed-From-Debian' & `Change-Type' fields.

I'll start adding README.gNewSense files to packages that don't have them yet. Samgee, you can also add the README.gNewSense files, if you have the time; that way we can get the job done sooner :)

-- rsiddharth

rsiddharth <rsd>
Group Member
Sun 22 Dec 2013 08:12:58 PM UTC, comment #12: 

While I was helping someone to submit a patch to us, there was some confusion about the Changed-From-Debian field. It was not clear from the wiki page for this person if the field could be multiple lines or just one. I think it would be good if you could specify that and make the example line a bit shorter so that it doesn't wrap.

I added a README.gNewSense to bacula by way of test. I see that not all of them have been added/modified. Are you still working on them or should I do them? It's all the same to me.

Sam Geeraerts <samgee>
Group administrator
Fri 20 Dec 2013 06:05:04 AM UTC, comment #11: 

I'll start updating the README.gNewSense files.

Thanks.

rsiddharth <rsd>
Group Member
Wed 18 Dec 2013 10:29:10 PM UTC, comment #10: 

Sorry, I must have misread. I thought the link field would be required in README.gNewSense.

I think you have write access to the bzr repo. If you're comfortable updating/creating the README.gNewSense files you can already start on it. Else I'll have a look at them in the weekend.

Sam Geeraerts <samgee>
Group administrator
Wed 18 Dec 2013 04:19:40 AM UTC, comment #9: 


> The link is the same format for all packages, so you could just > build that into the code. Then we won't need to maintain the
> More-Info-Link.


Yes, the link is inferred from the package name. Did you have a look at this page?

rsiddharth <rsd>
Group Member
Mon 16 Dec 2013 10:04:19 PM UTC, comment #8: 

That looks good. The link is the same format for all packages, so you could just build that into the code. Then we won't need to maintain the More-Info-Link.

You should be able to infer the change type from the bzr checkouts, but adding the Change-Type to README.gNewSense is probably a quicker win.

Sam Geeraerts <samgee>
Group administrator
Sun 15 Dec 2013 09:46:49 AM UTC, comment #7: 

I've updated the gns-deb-diff script.

It works as described here:

   http://www.gnewsense.org/sddhrth/gns-deb-diff-notes

If you feel that the format for the README.gNewSense is good, we can start updating the README files in the bzr repo.

Thanks.

rsiddharth <rsd>
Group Member
Sat 14 Dec 2013 03:24:01 PM UTC, comment #6: 

Thanks for the feedback samgee.

The 'Name:Value' format makes sense.

So, we may follow a format like this:


Changed-From-Debian: A short one line description about how this package differs from the corresponding Debian package.
Change-Type: `Added', `Removed' or `Modified'.
More-Info-Link: http://bzr.savannah.gnu.org/lh/gnewsense/packages-parkes/packag-name/annotate/head:/debian/README.gNewSense


in the README.gNewSense file, which can then be converted to:


||Source Package||Change Type||Reason||
||package-name||[http://bzr.savannah.gnu.org/lh/gnewsense/packages-parkes/packag-name/annotate/head:/debian/README.gNewSense|Modified]||A short one line description abou\
t how this package differs from the corresponding Debian package.||


I'll keep you posted on this.

rsiddharth <rsd>
Group Member
Fri 13 Dec 2013 08:24:20 PM UTC, comment #5: 

Looks good and it's one of the thought paths I had in mind when I pondered about this problem some time ago [1].

README.gNewSense can sometimes contain notes for the user or maintainer that are not about the freedom issues per se (e.g. linux-2.6). I agree that it would be good to have a one-liner in the file that we can use for the page. Debian commonly uses structured files in the form of "Name: Value", so my idea was to have a line like

Changed-From-Debian: Branding

That way it can appear anywhere in the file (though first line is a good place).

We can and should have README.gNewSense for every package we modify or add. For removed packages, the 2 obvious options are having just debian/README.gNewSense for it or putting the reason in a more structured blacklist. Either options seems a bit awkward, but the former is the easiest for you, so we'll just go with that until we think of something better.

Nice work so far. Please continue your efforts.

[1] http://lists.gnu.org/archive/html/gnewsense-dev/2013-01/msg00009.html

Sam Geeraerts <samgee>
Group administrator
Thu 12 Dec 2013 01:34:20 PM UTC, comment #4: 

I've started writing a script to automate the documentation of the difference between gNewSense & Debian.

Take a look: https://gitorious.org/gns-deb-diff/gns-deb-diff

I've attached a file, which contains the MoinMoin marked up table (it is not clean though) produced by the script.

Please read the README.md file & I'd like to hear your feedback.

Thanks.

(file #29886)

rsiddharth <rsd>
Group Member
Sun 08 Dec 2013 01:51:27 PM UTC, comment #3: 

Thanks samgee. I'll slowly start documenting the differences at the suggested URL.

And I'll use the format, you've suggested.

rsiddharth <rsd>
Group Member
Sat 07 Dec 2013 10:40:43 PM UTC, comment #2: 

I suggest using http://www.gnewsense.org/Documentation/3/DifferencesWithDebian.

I had thought about a simple format like the following, but you can use the Libreplanet one if you like.

||Package||Difference||
||GNOME Display Manager||Branding||
||XChat||Removed references to non-free software||

See MichaƂ's answer for information about our modifications.

Sam Geeraerts <samgee>
Group administrator
Mon 16 Sep 2013 01:14:01 AM UTC, comment #1: 

I'd like to start such page in the Wiki if it is not already created.

Can use a format similar to the one used in the Libreplanet wiki[1] that is used to list FSF non-free packages?

Where can I find a list of packages that were removed/modified in gNewSense?

Thanks.

[1]: http://libreplanet.org/wiki/List_of_software_that_does_not_respect_the_Free_System_Distribution_Guidelines

rsiddharth <rsd>
Group Member
Sun 01 Sep 2013 08:49:37 AM UTC, original submission:  

We should have a list of differences between Debian and gNewSense. It should show the reason for each removed or modified package, in a concise and user-readable way.

I would love for an automatic way to do this (i.e. from blacklist and README.gNewSense files), but I don't have a quick fix for that, so we'd better start with a manual list on the wiki.

Sam Geeraerts <samgee>
Group administrator

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attached Files
file #29886:  gns-deb-diff-table.txt added by rsd (2KiB - text/plain)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rsd (Posted a comment)
  • -email is unavailable- added by samgee (Submitted the item)
  •  

    Follow 3 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2013-12-12 rsd Attached File- Added gns-deb-diff-table.txt, #29886
    2013-12-07 samgee StatusNone In Progress
        Assigned toNone rsd

    Back to the top

    Powered by Savane 3.13-cf05.
    Corresponding source code