The need to show user generated content is growing on the web. However, different clients (mobile apps, or a web client) might need these images in other resolutions. Converting these every single time is time-consuming and inefficient.

Therefore this live image resizer, a joint project between inventid and Magnet.me, attempts to circumvent these issues.

A client can simply request an image, and specify the maximum height, width, and a format (e.g. /sfsdf_1040_1040.png). To support Apple's retina solution, this can be suffixed with an upscale parameter, e.g. /sfsdf_1040_1040_2x.png (but also _13x for future use). A quick check is made whether this image was previously scaled to that resolution.

If yes, a redirect is given to the cache location of that image (currently AWS S3). Otherwise the image is resized live, and served directly to the client, the cached version is uploaded to AWS S3.

Adding images is equally simple. A client can simply post an image, accompanied by a token. These tokens can be requested from a POST on /token (which you should firewall to certain IP's). That token is then valid once, so your client can upload the file directly, without having it to go through your own application (except for the identifier probably).

Logging takes place in a JSON Logstash enabled format, so it's easy to get into Logstash and Kibana. Great for logging!

You can simply request an image using a plain GET request to http://localhost:1337/example_100_100.png. This will trigger the server to search for the image with id example, and serve it in a PNG format in a 100x100 resolution. Depending on earlier requests, the image might be on the CDN (causing a redirect) or be transcoded on the fly and uploaded later. For Retina (or HiDpi) displays, the postfix _2x will appropriately resize the image to that size (or perform a redirect). Additional options can be send through the query parameter, such as ?fit=crop to crop the image to the bounding box you request.

In order to upload an image, you need to do a POST request to /token. This post has an payload of an id in json. This endpoint should generally be filtered out by your firewall or loadbalancer. The received token is valid for 15 minutes. The client can then directly use this token to upload a file.

An example command in curl is curl -vvv -XPOST http://localhost:1337/token -d '{"id": "test"}' -H "Content-Type: application/json"

The client uses another POST request to http://localhost:1337/someimage.jpg, this will cause the someimage key to be used. A token also should be send along, this is done in the HTTP-Headers in the X-Token parameter. The token will automatically expire once used. The token is only valid for one upload attempt and one id.

An example command in curl is curl -vvv -XPOST http://localhost:1337/test.jpg -H "X-Token: earlier-return-value" -F "image=@/home/user1/Desktop/test.jpg"

You need to copy the default.json.example to default.json in the config directory. Then, specify your own values.

In case you like to use it in production, call the script like this:

NODE_ENV=production node index.js

It will then additionally load the production.json file.

To keep the cache links, an additional Postgresql database is used. The program will auto create the tables and maintain the schema, using pg-migration. You can use a Docker container to run postgresql in development, or use the excellent postgres app for OSX. You need to create the database and its credentials yourself. After creating these, edit the default.json config file. An example for command line psql is:

 sudo -u postgres psql -c "CREATE USER imageresizer WITH PASSWORD 'rogierisgaaf';"
 sudo -u postgres createdb -E UTF8 -T template0 --locale=en_US.utf8 imageresizer
 sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE imageresizer to imageresizer;"

For quick saving, the original files are kept in images subdirectory (retrieving from AWS S3 to determine whether an image exists is too slow). Be sure to keep this data and backup it. You can also use the config to let it point to another directory. In that case, ensure the user can write there!

Due to the change from sqlite to postgresql, you will need to do a little migration:

1. Create a postgresql database on your server
2. Execute the first two migrations to create the database structure
3. Create the pg-migration tables CREATE TABLE dbchangelog(id bigint, datetime timestamp with time zone);CREATE UNIQUE index changeset on dbchangelog(id);
4. Add the first two ids to the table
5. Add the postgresql section to your config file
6. Disable uploading of images
7. Dump your current database sqlite3 /opt/live-image-resize/cache.sqlite .dump > /tmp/image.sql
8. Move this file to your database server. Delete any schema related lines (CREATE TABLE or CREATE INDEX).
9. Import the dataset by using cat /tmp/image.sql | sudo -u postgres psql imageresizer
10. Set the table owner to your user ALTER TABLE dbchangelog OWNER TO imageresizer; ALTER TABLE tokens OWNER TO imageresizer; ALTER TABLE images OWNER TO imageresizer;
11. Pull the new Docker container
12. Restart the container (the final changeset will be applied).
13. Re-enable your uploads

That's it!

Developing is relatively easy, once you know how it works. Since some programs are required for running the application, we recommend to develop using Docker (the rebuild is quite fast). On Linux and Windows, you will need to install Docker. On OSX the Docker Toolbox suffices.

1. After installing the Docker toolbox (which we will use here), you need to create a Docker machine docker-machine create inventid --driver=virtualbox
2. Then define the docker machine eval $(docker-machine env inventid)
3. Ensure you have a PostgreSQL instance available, see the section on Database on how to achieve this
4. Next (this also applies for Linux) we'll create the container mkdir -p /tmp/images && docker build --tag=test . && docker run -p 1337:1337 -v /tmp:/opt/images -v <YOUR_GIT_REPO_LOCATION>/config:/opt/live-image-resize/config test
5. Now you can start developing. After each change, stop the container (Ctrl-C) and re-execute the command again. Rebuilds of the container are relatively fast.

Quick way to send images (ensure you have jq installed)

IMAGE=test1234567
RES=`curl -vvv -XPOST http://192.168.99.100:1337/token -d "{\"id\": \"${IMAGE}\"}" -H "Content-Type: application/json"`
TOKEN=`echo $RES | jq -r .token`
curl -vvv -XPOST http://192.168.99.100:1337/${IMAGE}.jpg -H "X-Token: ${TOKEN}" -F "image=@/Users/Rogier/Downloads/878261728-5f264338.jpg"

You can use the Dockerfile to quickly stage stuff locally (on OSX use docker-machine).

If you have additions for the code, please fork the repo and open a Pull Request.