Compile and render Synapse's docs into a browsable, mobile-friendly and searchable website

[anoadragon453]

View the demo! https://anoadragon453.github.io/synapse/

Synapse's documentation has traditionally existed as a collection of notes inside our docs/ directory. While this has the benefit of being close to the code (such that it can be updated alongside code changes), it suffers from a lack of discoverability, searchability and presentation in general.

This PR attempts to organise our existing documentation into a searchable and user-friendly website for the benefit of both users of Synapse (as they can quickly browse and search for questions about running the homeserver), as well as Synapse developers in terms of quickly seeing what components of the system are documented, and which aren't.

For this project I chose mdbook. Partly because we're already using it for other Matrix-related projects, but also partly because of the extensibility of the UI and rendering, quick build times and great UX.

I've done by best to organise the changes into logical commits, with useful commit messages. As a basic overview of how mdbook works however:

• book.toml is a new file at the root of the repository that configures how mdbook behaves, the features of the site and where to look for the content.
• docs/SUMMARY.md defines the outline of the site (and is what is rendered into the left sidebar). When you add a new page, it needs to be linked to from this file.
• docs/website_files contains custom HTML, CSS and JS that's injected into the resulting static site files at build time. These files are defined in book.toml, and we utilise this extensibility to add a floating table of contents plugin, and to tweak some of the default UI.

This PR strictly keeps the old documentation where it was so that historical links don't break. That being said, ideally new documentation pages would land in a sensible location rather than the root of the docs/ directory. I've explained a little bit about the new structure (and what to do when adding a new page) in docs/README.md.

All that being said, there's still a few open-ended questions that I'd like to discuss on this pull request:

1. I've configured the CI through Github Actions to push the built documentation to the gh-pages branch via Github Actions. This means that the documentation will initially be available at https://matrix-org.github.io/synapse, and we could configure our DNS to point a more official-looking URL at it for the long term.

   The Github Actions configuration (located here) is currently configured to push to the latest version of the website on every new commit on develop.

   This is much easier to maintain than configuring CI to push to matrix.org's infrastructure, but does it sound reasonable to you? Note that mdbook supports versioning, but I've opted to keep that to a separate PR to minimise review scope.

2. I'd like to direct people to the website as much as possible, but appreciate that some people will still be going into the docs/ directory, either because they're not aware of the new site or just personal preference. mdbook is handy as the files are still just regular old markdown files.

   However, I appreciate that there are now several new folders living in the docs/ directory, and at present many of them only lead to a couple files (such as section headers, which don't make much sense if you're not looking at them on the website). I don't think these are much of a nuisance, but makes things slightly more confusing if you're just looking at codebase standalone.

3. I'm not sure what to do about the privacy policy templates in the docs/ directory.

I'd also like to note that while the current state of the website is quite nice, there's still a few warts:

• Some of our documentation is still in RST. I'd like to update these to be markdown, but doing so would break existing links. The most notable example is UPGRADE.rst - but perhaps we can just link to this file instead of hosting it on the docs site?
• The welcome and overview page is currently quite bare. I'd like to see some links to interesting parts of the docs for different types of visitors - such as developers versus sysadmins.
• mdbook supports having a button on each page that will take you to github to edit the page contents. Unfortunately, there's a bug where this feature does not produce the correct edit link if you've customised your documentation directory path (the default is src/, we're using docs/). Thus this feature is currently disabled in the config, but we can re-enable it once it's been fixed (upstream bug). fixed in mdbook 0.4.9!

For this reason I'm not putting a direct link to the documentation at the top of the root README yet, but hope to do so after we get the basic infrastructure in and refine things a little!

Requires: #10089.

[clokep]

This is much easier to maintain than configuring CI to push to matrix.org's infrastructure, but does it sound reasonable to you?

Seems reasonable to me, yes.

I'm not sure what to do about the privacy policy templates in the docs/ directory.

Seems like we could add a page describing what you need for a privacy policy and then include them, like we did for the sample config?

A couple of other thoughts:

• I think leaving the docs in place for now is the right thing, but we might want to slowly move them to "better" places. We might break some links, but that's what perma links are for.
• I don't know what to do about things like UPGRADE.rst since that's linked in MANY places, but we could possibly leave it and the link to the website showing the upgrade notes?
• I don't think we should hold this up on it being perfect, it will be easier to iterate on once it exists.

[clokep]

Seems like a great first cut to me! I'm sure there's improvements to make but we can probably do those bit by bit.

[clokep]

Should we put a link here to the rendered version or maybe we should add it to the release notes?

[anoadragon453]

Release notes sounds good - we'll need to flip the switch on the GitHub Pages settings once we've merged anyhow.