mkdocs setup (deploy, dev, user-guide) (webrecorder#375)

* Initial docs move * Setup mkdocs * Adds instructions for building docs * add new deployment docs, local and prod * set up three sections: deployment, dev and user guide * remove old deployment docs * ci: mkdocs gh-pages publish Co-authored-by: sua yoo <[email protected]> Co-authored-by: Ilya Kreymer <[email protected]>
urbanlab · Dec 6, 2022 · a74d88d · a74d88d
1 parent 82ffc0d
commit a74d88d
Show file tree

Hide file tree

Showing 12 changed files with 283 additions and 120 deletions.
diff --git a/.github/workflows/docs-publish.yaml b/.github/workflows/docs-publish.yaml
@@ -0,0 +1,19 @@
+name: docs-publish
+on:
+  push:
+    branches:
+      - main
+      - docs!
+permissions:
+  contents: write
+
+jobs:
+  deploy_docs:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: 3.x
+      - run: pip install mkdocs-material 
+      - run: mkdocs gh-deploy --force
diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml
@@ -36,6 +36,7 @@ jobs:
         uses: docker/build-push-action@v3
         with:
           context: backend
+          platforms: linux/amd64,linux/arm64
           push: true
           tags: ${{ env.REPO_PREFIX }}webrecorder/browsertrix-backend:${{ env.VERSION }}, webrecorder/browsertrix-backend:latest
           cache-from: type=gha,scope=backend
@@ -46,6 +47,7 @@ jobs:
         uses: docker/build-push-action@v3
         with:
           context: frontend
+          platforms: linux/amd64,linux/arm64
           push: true
           build-args: |
             VERSION=${{ env.VERSION }}

diff --git a/Deployment.md b/Deployment.md
diff --git a/README.md b/README.md
@@ -14,21 +14,27 @@ The system is designed to run in both Kubernetes and Docker Swarm, as well as lo
 
 See [Features](https://browsertrix.cloud/features) for a high-level list of planned features.
 
-## Deployment
-
-See the [Deployment](Deployment.md) page for information on how to deploy Browsertrix Cloud.
-
 
 ## Development Status
 
-Browsertrix Cloud is currently in an alpha stage and not ready for production. This is an ambitious project and there's a lot to be done!
+Browsertrix Cloud is currently in an early beta stage and not fully ready for production. This is an ambitious project and there's a lot to be done!
 
 If you would like to help in a particular way, please open an issue or reach out to us in other ways.
 
+## Documentation
+
+Docs are available at: [https://docs.browsertrix.cloud/](https://docs.browsertrix.cloud/) created from the markdown in the [./docs](./docs) on the main branch.
+
+To build the documentation locally, install Material for MkDocs with pip:
+
+```shell
+pip install mkdocs-material
+```
+
+In the project root directory run `mkdocs serve` to run a local version of the documentation site.
+
 ## License
 
 Browsertrix Cloud is made available under the AGPLv3 License.
 
 If you would like to use it under a different license or have a question, please reach out as that may be a possibility.
-
-
diff --git a/docs/deploy/index.md b/docs/deploy/index.md
@@ -0,0 +1,14 @@
+# Introduction
+
+Browsertrix Cloud is designed to be a cloud-native application running in Kubernetes.
+
+However, despite the name, it is perfectly reasonable (and easy!) to deploy Browsertrix Cloud locally using one of the many available local Kubernetes options. Here are a few recommendations for different scenarios.
+
+The main requirements for Browsertrix Cloud are:
+
+- A Kubernetes Cluster
+- Helm (package manager for Kubernetes)
+
+We have prepared a [Local Deployment](./local) and [Production (Self-Hosted and Cloud) Deployment](./production) guides to help with
+setting up Browsertrix Cloud for different scenarios.
+
diff --git a/docs/deploy/local.md b/docs/deploy/local.md
@@ -0,0 +1,82 @@
+# Local Deployment
+
+To just test out Browsertrix Cloud on your local machine, you can use many of the single-node Kubernetes tools.
+
+Browsertrix Cloud has been tested with Docker Desktop (with Kubernetes enabled), microk8s, minikube and k3s.
+
+Here are a few different ways to get started with Browsertrix Cloud. The instructions below assume you have cloned
+the [https://github.com/webrecorder/browsertrix-cloud](https://github.com/webrecorder-browsertrix-cloud) repository locally,
+and have local package managers for your platform (eg. `brew` for mac) already installed.
+
+After following the environment-specific instructions below, simply run: 
+
+`helm upgrade --install -f ./chart/values.yaml -f ./chart/local-config.yaml btrix ./chart/`
+
+The local setup includes the full Browsertrix Cloud system, with frontend, backend api, db (via MongoDB) and storage (via Minio)
+
+An admin user with name `[email protected]` and password `PASSW0RD!` will be automatically created.
+
+These settings can be changed in `charts/examples/local-config.yaml`. Note that the admin user and password will not be reset after creation.
+
+To access the server load: [http://localhost:30870/](http://localhost:30870/) in your browser.
+
+Here are some additional environment-specific deployment specific instructions:
+
+### Docker Desktop -- Mac
+
+For Mac, we recommend testing out Browsertrix Cloud on Docker Desktop as that will be one of the simplest options.
+
+To run Browsertrix Cloud on Docker Desktop:
+
+1. Ensure `Enable Kubernetes` is checked from the Preferences screen.
+
+2. Install [Helm](https://helm.sh/), which can be installed with `brew install helm` or [other options](https://helm.sh/docs/intro/install/)
+
+3. Run the Helm command as described above.
+
+### MicroK8S
+
+For Ubuntu and other linux distros, we recommend using MicroK8S for both local deployment and production.
+
+1. Install MicroK8s, by running `sudo snap install microk8s --classic` [see more detailed instructions here](https://microk8s.io/docs/getting-started) or [alternate installation instructions here](https://microk8s.io/docs/install-alternatives)
+
+2. Install the following addons `microk8s enable dns hostpath-storage registry helm`. (For production, also add `ingress cert-manager`)
+
+3. Wait for add-ons to finish installing with `microk8s status --wait-ready`
+
+4. Run the Helm command as described above, prefixed with `microk8s`, eg. `microk8s helm ...`
+
+### Minikube
+
+1. Install Minikube [following installation instructions](https://minikube.sigs.k8s.io/docs/start/), eg. `brew install minikube`
+
+2. Run the Helm command as described above.
+
+
+## Waiting for Cluster to Start
+
+Especially on first run, it may take a few minutes for the Browsertrix Cloud cluster to start, as all images need to be loaded.
+
+You can try running the command: `kubectl wait --for=condition=ready pod --all` to wait for all pods to be initialized.
+
+If this command fails, you can also run `kubectl get pods` to see the status of each of the pods.
+
+There should be 4 pods listed: backend, fronend, minio and mongodb. If any one is not ready for a while, something may be wrong.
+
+### Debugging Pod Issues
+
+To get more details about why a pod has not started, you can run `kubectl describe <podname>` and see the latest status at the bottom.
+
+Often, the error may be obvious, such as failed to pull an image.
+
+If the pod is running, or previously ran, you can also get the logs from the container by running `kubectl logs <podname>`
+
+The outputs of these commands will be helpful if you'd like to report an issue [on GitHub](https://github.com/webrecorder/browsertrix-cloud/issues)
+
+
+## Uninstalling
+
+To uninstall, run `helm uninstall btrix`.
+
+By default, the database + storage volumes are not automatically deleted. To fully delete all persistent data created in the cluster, also run `kubectl delete pvc --all`.
+
diff --git a/docs/deploy/production.md b/docs/deploy/production.md
@@ -0,0 +1,70 @@
+# Production: Self-Hosted and Cloud
+
+For production and hosted deployments (both on a single machine or in the cloud), the only requirement is to have a designed domain
+and (strongly recommended, but not required) second domain for signing web archives. 
+
+We are also experimenting with Ansible playbooks for cloud deployment setups.
+
+The production deployments also allow using an external mongodb server, and/or external S3-compatible storage instead of the bundled minio.
+
+
+## Single Machine Deployment with MicroK8S
+
+For a single-machine production deployment, we recommend using microk8s.
+
+1. Install MicroK8S, as suggested in [the local deployment guide](./local) and ensure the `ingress` and `cert-manager` addons are also enabled.
+
+2. Set the `ingress.host`, `ingress.cert_email` and `signing.host` fields in `./examples/microk8s-hosted.yaml` to your host and domain
+
+3. Set the supeadmin username and password, and mongodb username and password, also in `./examples/microk8s-hosted.yaml`
+
+4. Run with `helm upgrade --install -f ./chart/values.yaml -f ./chart/microk8s-hosted.yaml btrix ./chart/`
+
+
+### Using Custom Storage
+
+If you would like to use existing external storage, such an existing S3-compatible storage, also set the default storage, for example:
+
+```
+minio_local: false
+
+storages:
+  - name: "default"
+    access_key: <access key>
+    secret_key: <secret key>
+
+    endpoint_url: "https://s3.<region>.amazonaws.com/bucket/path/"
+```
+
+Note that this setup is not limited to Amazon S3, but should work with any S3-compatible storage service.
+
+
+### Using Custom MongoDB
+
+If you would like to use an externally hosted MongoDB, you can add the following config to point to a custom MongoDB instance.
+
+The `db_url` should follow the [MongoDB Connection String Format](https://www.mongodb.com/docs/manual/reference/connection-string/)
+which should include the username and password of the remote instance.
+
+
+```
+mongo_local: false
+
+mongo_auth:
+  db_url: mongodb+srv://...
+
+
+## Cloud Deployment
+
+There are also many ways to deploy Browsertrix Cloud on various cloud providers.
+
+To simplify this process, we are working on Ansible playbooks for setting up Browsertrix Cloud on commonly used infrastructure.
+
+Thus, far we have the following Ansible playbooks available:
+
+### Digital Ocean
+
+*TODO: Add more details*
+
+
+
diff --git a/frontend/README.md → docs/dev/frontend-dev.md b/frontend/README.md → docs/dev/frontend-dev.md
@@ -1,7 +1,12 @@
-# Browsertrix Cloud frontend
+# Running the Frontend
+
+This guide explains how to deploy an instance of the Browsertrix Cloud frontend for development.
+The frontend can connect to a Browsertrix Cloud API backend running locally or remotely.
 
 ## Quickstart
 
+Ensure the current working directory is set to the `/frontend` folder.
+
 Install dependencies:
 
 ```sh
@@ -20,6 +25,12 @@ Update `API_BASE_URL` in `.env.local` to point to your dev backend API. For exam
 API_BASE_URL=http://dev.example.com/api
 ```
 
+If connecting to a local deployment cluster, set API_BASE_URL to:
+
+```
+API_BASE_URL=http://localhost:30870/api
+```
+
 Start the dev server:
 
 ```sh