All notable changes to this project should be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
http
: Rewrite specific URL patterns to DBpedia resource URIs:http://<locale>.dbpedia.org/page/<slug>
https?://<locale>.wikipedia.org/wiki/<slug>
http
: Allow lookup of multiple URI clusters at once with theuris
parameter.http
: log in the Uvicorn logger that app startup is ready.
- Install the latest RocksDB release from GitHub archive instead of git (see #6).
- Update python to v3.7 and install the latest version of python packages.
- Added
mypy
to development dependencies.
db
: lock contention (RocksIOError
, see #8), fixed by reusing the admin DB connection inrestore.create_backup()
.
http
: purge old data DBs on Gunicorn/Uvicorn initialization.
http
: let Gunicorn preload the app (instead of per worker).
loader
: ability to restore from backup by registering them in the admin DB.loader
: CLI to interactively select a backup to restore.
loader
: hot-swap DB when the current DB never completed loading.
db.get_connection()
: RocksDB API change.
loader
: find and fetch latest snapshot via Databus SPARQL query (see #5).
http
: Gunicorn performance tuning.loader
: RocksDB performance tuning.
http
: movemeta
info to its own top-level key.- Docs: updated readme.
loader
: don't overlook decompressed chunks that are smaller than one line.loader
: only split on the secondary seperator once.
- Docs: deployment and maintenance notes.
parser
: parse RDF literals from ntriple lines.loader
: progress bars.http
: include singleton IDs and meta info in response (see #1).
- Adjust DBp Global namespace to global.dbpedia.org/id/.
loader
: only load and backup if a new snapshot is available.loader
: load compressed TSV snapshot instead of ntriples part files.loader
: separate concerns with producer/consumer pattern.http
: replace ApiStar with Starlette.- Restructured Docker configuration.
- pinned python version at 3.6.
loader
: close the event loop after load and backup are done.
- Docs: installation and usage notes.
loader
: only load and backup if new parts are available.
- Docker image path.
loader
: close the event loop after load and backup are done.
http
: run with Gunicorn server in production.loader
: don't attempt manual compaction (prone to stalling).
loader
: admin part key encoding.
- URI cluster
loader
from Spark output. http
interface based on ApiStar.