The following is the GNU All-permissive License as recommended in https://www.gnu.org/licenses/license-recommendations.en.html

Copyright (C) 2024 Free Software Foundation sysadmin@fsf.org

Copying and distribution of this file, with or without modification, are permitted in any medium without royalty provided the copyright notice and this notice are preserved. This file is offered as-is, without any warranty.

Contributions are welcome. See https://savannah.gnu.org/maintenance/fsf/.

Discourse

[[!toc levels=5]]

WARNINGS

• do NOT enable the sso overrides email option.

Documentation for admins

SSO settings note

Do not enable sso overrides email, because users can change their my.fsf.org / CAS email address without validation. It's important that users are forced to validate email addresses upon signup or editing their Discourse account, otherwise accounts will get merged based on email addresses. Before you consider changing this on the live forum, do thorough testing on the dev forum and talk to Andrew. Note that you'll have to change the email address on two accounts on my.fsf.org to move an email address from one account to another on my.fsf.org / CAS.

Running psql to inspect database

docker container ls # get container id, c3ac4fbfe7e3 in next command
docker exec -it c3ac4fbfe7e3 /bin/bash -c 'su -c "psql -d discourse" postgres'

Example of searching for uses of a keyword, ip_add, in fields in the db

docker exec -it c3ac4fbfe7e3 /bin/bash
su postgres
for t in $(echo '\x on;\dt'|psql -d discourse | sed -nr 's/^Name.*\s(\S+)/\1/p'); do for f in $(echo "\d+ $t"|psql -d discourse | sed -nr 's/^\s*(\w*ip_add\w*).*/\1/p'); do echo $t - $f; echo "select count(*) from $t where $f is not NULL;"|psql -d discourse; done; done

Theme customization settings

The theme customization editor won't let you copy more than one line out of the editor. So, here are the edits:

CSS:

#fsf-footer-text { text-align: right; padding: 20px; }

Footer:

<div id="fsf-footer-text">
  <p><a href="https://www.fsf.org/about/dmca-notice">Copyright Infringement Notification</a></p>
  <p>Source code:
    <a href="https://github.com/discourse/discourse">#1</a>,
    <a href="https://weblabels.fsf.org/forum.members.fsf.org/CURRENT/" rel="jslicense">JS Licenses</a>
    <a href="https://github.com/eriko/discourse_cas_sso">#3</a>,
    <a href="https://weblabels.fsf.org/forum.members.fsf.org/CURRENT/" rel="jslicense">JS Licenses</a>
  </p>
</div>

Installing discourse

DNS

enter something like this. check that the ipv4 address points to eggs, and you should configure eggs.gnu.org as the smtp server below.

MX      10 forum.members.fsf.org.
TXT     v=spf1 a mx ip4:208.118.235.92 ~all

Ansible

add the vm to the vm, behind-http-proxy, ufw, nginx, discourse, docker, and rvm groups. check other recent vms for other groups. note that forum0d is a member of some unnecessary groups.

After you run ansible for the first time, add it to the office-only-web group until the site is ready for use by others.

Setting up the environment

Git repo: https://github.com/discourse/discourse

Instructions come from here: https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md

https://github.com/discourse/discourse_docker.git should be cloned as /var/discourse/

To change existing low level site configuration variables, edit /var/discourse/containers/app.yml and run ./launcher rebuild app. To generate app.yml for the first time, stop nginx, run ./discourse-setup, enter eggs.gnu.org for the smtp address (localhost won't work), enter "" answers for SMTP user name and password, enter 25 as port number, provide sysadmin@gnu.org,yourname@fsf.org as admin emails, skip LE setup, then edit app.yml.

add something like these configs under env: in containers/app.yml (note the forum domain name):

  # -- sudoman 2018-06-01
  HTTP_PROXY: http://serverproxy0p.fsf.org:8118
  HTTPS_PROXY: http://serverproxy0p.fsf.org:8118
  NO_PROXY: localhost,127.0.0.0/8,forum.members.fsf.org

comment out the expose: section.

add this line to the templates: section:

  - "templates/web.socketed.template.yml" # -- sudoman 2018-06-01

change the smtp password to "".

if this is your first time building the image, run ./launcher bootstrap app && ./launcher start app, otherwise, run ./launcher rebuild app.

Look at the nginx configurations on forum0p.members.fsf.org, or whichever vm hosts discourse, to see the socket configuration for talking with discourse.

Comment out the XSS config in /etc/nginx/conf.d/ (Discourse sets its own) and set the CSP config to something like:

add_header Content-Security-Policy "default-src 'self'; style-src 'self' 'unsafe-inline'; script-src 'self' 'unsafe-inline' 'unsafe-eval';";

Initial setup up via the web interface

Go to your domain's URL in a browser and follow along with the steps. Your user name should be you CAS user name. It's pretty straight-forward, but when it asks you for "Your community’s name", you should put in something like "The Free Software Foundation Associate Member Forum", not "The FSF". The forum should be marked private.

Installing plugins

https://meta.discourse.org/t/install-plugins-in-discourse/19157

Using ./launcher enter app then cloning the repo in /var/www/discourse/plugins manually inside the image causes the data to be lost after running ./launcher rebuild app.

Installing and setting up cas sso web service

The developer of the extension says not to use this code base. We should move to SAML+CAS in the future.

You need to install an older version of ruby. add the server to the 'rvm' group in ansible and then run rvm autolibs disabled then rvm install 2.3 and something like rvm use 2.3.7.

Clone https://vcs.fsf.org/git/discourse_cas_sso.git under /srv/discourse_cas_sso as the discourse user. Use the fsf-config branch.

If these instructions are incomplete, instructions can be found here: https://github.com/eriko/discourse_cas_sso

Copy the random secret password from /etc/discourse/ and add it to the admin section of the site. (Search the admin pannel for "sso secret".)

Update the configurations as described in the instructions. Edit configs under /etc/discourse/ via ansible. The sso.secret value (set via ansible) must match the "sso secret" value in Discourse. require_activation should be set to true.

export HTTP_PROXY=http://serverproxy0p.fsf.org:8118; export HTTPS_PROXY=http://serverproxy0p.fsf.org:8118; export NO_PROXY=localhost,127.0.0.1

cd /srv/discourse_cas_sso/

bundle install
bundle exec rake --tasks

RAILS_ENV=production bundle exec rake db:migrate

chown -R discourse:discourse /srv/discourse_cas_sso/

Start the server with systemctl start discourse_cas_sso.

Enabling / disabling SSO

Disabling

If you need to disable SSO due to brokenness, see "What if you check it by mistake?" at the top of the page: https://meta.discourse.org/t/official-single-sign-on-for-discourse-sso/13045

cd /var/discourse
./launcher enter app
rails c
irb > SiteSetting.enable_sso = false
irb > SiteSetting.enable_local_logins = true
irb > exit
exit

Enabling

confirm all new settings as you make them:

• enable invite only
• enable login required
• set "sso url" to something like https://forum0p.members.fsf.org/sso
• set "sso secret" to secret value in /srv/discourse_cas_sso/config/configatron/production.rb or vice versa
• set "logout redirect" to someting like https://cas.fsf.org/logout?service=https://forum0p.members.fsf.org
• should enable user name override
• disable "enable local logins"
• do NOT enable email overrides. this would break account identities and cause accounts to get mixed up because we don't validate email addresses in CAS / my.fsf.org.
• select "enable sso" (note that if using the proper discourse_cas_sso config, emails don't need verification by the external site.

before you log out of that account or try to log in, you need to log in as another user in a private window and then to grant that user admin access from your open login. then in the private window, delete the account you used to set up the site, because it doesn't have cas credentials associated with it. then restart your private session and log in with the user name you used to set up the site. then revoke admin access from the second user account.

Debugging cas sso

edit /srv/discourse_cas_sso/config/environments/production.rb and set config.log_level = :info. Make sure to revert this and to restart the service after you are done debugging, because it logs private information like user names and ip addresses. At the time of writing, log rotation is not set up for this file.

Setting up email reply delivery

exim is configured via the discourse ansible role to listen on port 26 so that the discourse postfix service can listen on port 25. local delivery to the world still works even with the non-standard port.

https://meta.discourse.org/t/straightforward-direct-delivery-incoming-mail/49487

# if you don't have a mail-receiver config, copy it from /var/discourse/samples/ and edit.
vim /var/discourse/containers/mail-receiver.yml

change the mail domain, the discourse enpoint, add an API key generated from /admin/api/keys, add something like this to the env: section (note the forum domain name):

  # -- sudoman 2018-09-19
  HTTP_PROXY: http://serverproxy0p.fsf.org:8118
  HTTPS_PROXY: http://serverproxy0p.fsf.org:8118
  NO_PROXY: localhost,127.0.0.0/8,forum.members.fsf.org

then run these commands:

./launcher bootstrap mail-receiver

./launcher start mail-receiver

make an mx record for the recipient domain. make sure to update the mx record when migrating hosts.

configure reply by email enabled, reply by email address with a value of something like replies+%{reply_key}@forum.members.fsf.org, and manual polling enabled.

Add a config like this to mail.fsf.org:/etc/exim4/conf.d/routers.conf near an existing forum conf:

forum0d_members_fsf_router:
  driver = manualroute
  domains = forum0d.members.fsf.org
  transport = remote_smtp
  route_list = * forum0d.members.fsf.org

useful commands:

./launcher logs mail-receiver

./launcher enter mail-receiver
mailq
exit

Other settings

set the backup frequency to be once per day, instead of once every seven days. enable "force https" once you're sure that you are ready for that. disable "automatically download gravatars".

Upgrading

make sure to take a database snapshot before upgrading.

Via the web browser

Don't use this method until we get the removal of the restricted repos merged upstream. If we upgrade via the web, it might try a git pull on /var/discourse that will fail due to a merge conflict with our code. When that happens, we need to build our own patched version of the base docker image and use that.

If the web upgrade fails, run ./launcher start app or ./launcher rebuild app from the command line in /var/discourse to get the app started again, then try to do an upgrade via the cli.

To upgrade via the web interface, visit /admin/upgrade in a web browser.

Via the command line

To begin upgrading Discourse and to check whether we need to build a new Docker base image, do the following on your local machine:

### First time only
git clone https://github.com/discourse/discourse_docker
cd discourse_docker/
git remote add fsf git@vcs.fsf.org:discourse_docker.git
git pull fsf

### Do the following every time
git checkout main # not master
git pull

git checkout fsf-master
git pull
git merge main # not master

## Fix conflicts, add major changes to the FSF-CHANGELOG.mdwn

git add -p
git commit

git push

Check image/base/slim.Dockerfile to see whether it adds any non-free repos, and ensure that Debian is still the base image (check top of that file). If either have changed, we need to figure out whether the software will work without the non-free bits, and potentially edit our fsf-master branch, and build a new base image. (Skip to section about building a new base image below.)

If there were no merge errors about /var/discourse/launcher containing the line image=discourse/base:build instead of the default value, go ahead with the normal upgrade procedure. Otherwise, you'll need to build a new Docker base image first. (Skip to that section further below.)

If there is a new entry for image=discourse/base:..., then pre-fetch the Docker images before taking the site offline:

ssh root@forum.members.fsf.org

cd /var/discourse
git checkout fsf-master
git pull

docker pull $(grep -E "^image=\"discourse/base" launcher | grep -E -o "discourse/base:[0-9.-]+")

You may also want to run:

docker pull discourse/mail-receiver:release

Make a backup of the database by making a snapshot under the admin web interface.

Make an out of band notice. Example: https://hostux.social/@fsfstatus/102882143256592757

SSH to forum.members.fsf.org

ssh root@forum.members.fsf.org

When taking Discourse offline for a ./launcher rebuild app step, not when building a new base image, it's good to set up the site-wide 503 error page. Make sure to revert your changes when done.

cd /etc/nginx/sites-enabled/
rm -f *-ssl.conf
ln -s ../sites-available/503-everything.conf
service nginx reload

To upgrade Discourse:

cd /var/discourse
git checkout fsf-master
git pull

./launcher rebuild app

Answer n to the question about recovering space.

Would you like to attempt to recover space by cleaning docker images and containers in the system?(y/N)n

Then update the mail receiver:

./launcher rebuild mail-receiver

Now add the hard-coded line that adds a link to the weblabels, and replace text about non-free browsers:

./launcher enter app

patch -p1 << EOF
diff --git a/app/views/layouts/application.html.erb b/app/views/layouts/application.html.erb
index 9cb5efcce9..5003d5e57c 100644
--- a/app/views/layouts/application.html.erb
+++ b/app/views/layouts/application.html.erb
@@ -96,6 +96,8 @@
     <section id='main'>
     </section>

+    <div style="display: none"><a href="https://weblabels.fsf.org/forum.members.fsf.org/CURRENT/" rel="jslicense">JS Licenses</a></div>
+
     <% unless current_user %>
       <form id='hidden-login-form' method="post" action="<%=main_app.login_path%>" style="display: none;">
         <input name="username" type="text"     id="signin_username">

EOF

grep -ril https://browsehappy.com . | xargs -i sed -i -e 's:https\://browsehappy.com:https\://directory.fsf.org/wiki/Category/Internet-application/web-browser:g; s: href="https\://www.discourse.org/faq/#browser"::g' {}

exit

./launcher restart app   ## not 'rebuild'!

At the time of writing, the configs you should re-enable when done are:

forum0p.members.fsf.org.conf
forum0p.members.fsf.org-discourse-docker-proxy-ssl.conf
forum0p.members.fsf.org-ssl.conf

Use these commands to bring the site back up:

cd /etc/nginx/sites-enabled/
rm -f 503-everything.conf
ln -s ../sites-available/*ssl* .
service nginx reload

Open the https://forum.members.fsf.org/ up in a private browser window CTRL+SHIFT+P and test the login.

Once the forum is up and running, and if everything is working properly, run the following command to remove images that are not currently running: support thread

./launcher cleanup

This step may take less than 13 minutes.

Now make sure that the LibreJS weblabels are up to to date. If not, add entries to the git@vcs.fsf.org:weblabels.fsf.org.git repo.

Make an out of band notice. Example: https://hostux.social/@fsfstatus/102882143256592757

Update the weblabels for the FSF forum (see below).

Weblabels updates

I suggest copying and pasting the new, rejected JS file names over the existing file names in the weblabels repo. Then do a git diff to see a summary of what has changed. Look under each diff entry to see the .tgz file that needs to be updated. unpack the original tgz files and look at what is inside of them. clone the discourse app repo, and check out the used git version according to the forum admin page. run a find command in the discourse app repo for one of those files in the unpacked tgz. look at what is in its parent folder. if its contents matches, that's one of the tgzs you need to create and copy into the weblabels repo. copy them over, clean out the temporary tgz unpack directories, then commit and push. visit the weblabels page to refresh the js list cache for the plugin. visit the forum to see what else needs fixing. rinse and repeat.

Building a new Docker base image

If the base image is still Debian based, and adds no non-free repos / programs, and the base image in the launcher file is like this: image="discourse/base:2.0.20190625-0946", but with a date after 2019-08-23, then we may be able to continue using the upstream image, and skip building a new image. Otherwise, we should add in image=discourse/base:build, fix freedom issues, and build the image, as described below. Note that major changes to the docker image may break the forum, so this should be talked about with the team before making such changes.

If merging the master branch into fsf-master creates a conflict with the code that removes the restricted repo, or sets the Docker base image name in the launcher script, then we need to build a new local docker image with our local changes.

Stopping Discourse and setting up the 503 page is not necessary for this step of the upgrade.

While the forum is up and running, run the following command to remove images that are not currently running: support thread

./launcher cleanup

Then proceed with building the new base image.

cd /var/discourse
git checkout fsf-master
git pull

git log # to check for your recent commit

/var/discourse/image/base/Dockerfile should not contain the word restricted.

Make sure that /var/discourse/launcher contains the line image=discourse/base:build instead of the default value.

Building a new base image will take a long time, so it should be done while the previous version of Discourse is still running.

cd /var/discourse/image/
ruby auto_build.rb base

This step takes about an hour.

Continue with the instructions above.