Skip to content

Latest commit

 

History

History

Backend

README.md

Charter—backend

This is a happy-go-lucky collection of (mostly Node) scripts that run the backend for the Charter iOS app.

Getting started

This isn’t the most user-friendly process.

Overview

Set up

I’ve been running the backend on DigitalOcean, but it should work anywhere.

Requirements

  • node, >4.0
  • npm
  • forever (from npm)
  • MongoDB
  • Java 8
  • RESTHeart

Getting it running

The general idea behind the backend is this: import archival data from lists.swift.org into a MongoDB database. Then set up a mail listener that’ll listen for new emails from the mailing lists and add those to the database. Then start up the web server (which at this stage is just a REST wrapper around MongoDB).

  1. Install the requirements listed above
  2. If you want realtime updates, make sure you’re subscribed to the lists and will receive emails to your server (you can run the server without doing this though—the data will be constant)
  3. Check out the codebase with git clone https://github.com/matthewpalmer/Charter
  4. Open up the Backend directory
  5. Generate JSON from the archives hosted on the Swift Mailing List website, ./run.sh ~/Desktop/list-data ~/Desktop/list-json
  6. Import that JSON into MongoDB: mongoimport --db charter --collection emails --type json --file swift-evolution.json --jsonArray --upsert (where swift-evolution.json is the JSON file generated by step 5. Repeat for each list in the list-json directory). Create indexes on the database for date, mailing list, and a text index for content, from, and subject (db.emails.createIndex({from: "text", content: "text", subject: "text"}, {weights: {from: 50, subject: 50}}))
  7. Run the database reformatting script, node db-format.js (you can ignore most of the output from this script)
  8. Run the mail listener script with forever start mail.js
  9. Run the web server with nohup java -server -jar ~/restheart-1.1.5/restheart.jar ~/restheart-1.1.5/etc/restheart.yml &

Notes to myself

Subscribe to the lists

Make sure you’re actually subscribed to the Swift mailing lists with DNS set up so that you’ll receive the messages to your server instance

Install forever

sudo npm install -g forever

RESTHeart config

Adapted from the RESTHeart docs.

Create a user that has read permission on the emails database to use with RESTHeart.

> use admin
> db.createUser({
    user: "db_username",
    pwd: "db_password",
    roles:[ {role: "read", db: "charter" }]
})

Set the following line in restheart/etc/restheart.yml to connect with that user

mongo-uri: mongodb://db_username:[email protected]/?authSource=admin

Add some HTTP Basic Auth security (note that this is different to the database user you set up above) in restheart/etc/security.yml.

## Configuration for file based Identity Manager
users:
    - userid: some_REST_user
      password: some_REST_password
      roles: [users]

# Users with role 'users' can GET any collection or document resource (excluding dbs)
permissions:
    - role: users
      predicate: regex[pattern="/.*/.*", value="%R", full-match=true] and method[value="GET"]

Start the server with the config file: java -server -jar restheart.jar etc/restheart.yml