Koype

From IndieWeb

Koype is an open-source project that serves as an extensible social hub with support for Micropub, Webmention, IndieAuth, WebSub and other IndieWeb and Web standards.

Features

Until Koype has its own website up and running (or if Jacky Alciné chooses to use his own site to measure progress), this Wiki page will serve as a way to measure progress.

Pagination

Koype makes use of different forms of pagination, namely date-based pagination and a form that sorts by a determined lexicographic order on a particular field.

  • For date-based pagination, it'll take in any ISO8601/RFC3339 timestamp as a cursor and use that with limit, offset and other filters to walk over a time range.
    • There's work to allow duration-centric values to be used in limit and offset as well.
  • For lexicographic sorting, it'll internally pick a field based on the item's type and use that for sorting. This is mainly used for things like categories and syndication targets

Available

Async reply context hydration

When a URL is encountered when creating a new post, Koype runs it through its set of URL transformers. Depending on the authority of a URL (here being the host name and port of a URL), Koype will attempt to fetch a normalized (formatted in MF2+JSON) representation of the URL. This is built to work with XRay and can work with any other tool that returns MF2+JSON either as the root object, under a specified key or returns an HTML page formatted with MF2 with an item that has a property named url that contains the URL to be parsed.

In the event that no MF2 can be resolved from the remote URL, OGP, Twitter Card and the HTML is parsed to a minimum viable constraint to permit it as a Microformat item usable as a reply context. The resulting parser approach is hinted in the type of the resulting object.

Async Webmention processing

Koype should be used with a Webmention service that supports callbacks. That allows Koype to lean into a push-centric approach on collecting incoming Webmentions and parsing the result of outgoing Webmentions. If the callback hasn't been called within a configured set amount of time, Koype will fall back to fetching Webmentions for pages in regard to their last q=source request within a preconfigured interval. By default, if no incoming Webmention callback has been called in 12 hours, Webmentions for pages are fetched once every two hours (planned and tracked at https://git.jacky.wtf/indieweb/koype-next/issues/32).

Planned

Shared Access

With specific scopes to Koype, one can use a token to edit resources on it. This would allow for automatic editing of posts using Micropub from remote services. This would be done by adding a parameter to the Micropub endpoint URL that'll tell it to validate the token using a different endpoint, one to be determined by Koype. The Micropub endpoint would be provided when one fetches the specific page and will be available for a period of time for use in the browser, none is available via q=source.

The custom endpoint will check if the provided token is valid from its issuer, matching the place we've issued a token for specific people outside the content's owner. If they're a known member of the URL's audience, then other operations will be available to them. Koype will also apply per-identity rules on content availability, with overriding support. This can allow for a stricter ACL on who can see what on a local level and making URLs themselves transparent in what information they share (in terms of url design).

This is being tracked at https://git.jacky.wtf/indieweb/koype-next/issues/62

Address Book

Koype builds a nickname cache that's usable for quickly mentioning people in the system. It's available with authentication using the Micropub query ?q=contact. There's also going to be logic to automatically hydrate and update contacts using WebSub.

This is being tracked at https://git.jacky.wtf/indieweb/koype-next/issues/17.

Content Removal

Inspired by a post about the ethics of Webmentions, one approach to helping this is providing an extension to Micropub that allows for one to query for all items that reference a particular URL. This can be then used to make a bulk visibility or deletion of content on one's site.

This is being tracked at https://git.jacky.wtf/indieweb/koype-next/issues/63.

File System

Koype works on a virtual filesystem. The long-term goal is to allow for it to write to object storage, come preloaded with a sample filesystem for demonstration purposes and simplifying backups. Koype works with an idea of resources, a way to organize files on a filesystem. For example, all entries are bundled under a 'entries' resources, contacts under 'contacts', and so on. The following is an example of what the layout of files would look like: 

❯ tree data
data
├── categories
│   └── 2022
│       ├── 2
│       │   ├── b3
│       │   │   └── b36292k2
│       │   └── ji
│       │       └── ji0af7h0
├── channels
│   └── 2022
│       ├── 2
│       │   ├── 12
│       │   │   └── 125d49c7
│       │   ├── 5l
│       │   │   └── 5lc3hb39
├── db
│   ├── koype.sqlite
│   ├── koype.sqlite-shm
│   └── koype.sqlite-wal
├── destinations
│   └── 2022
│       └── 2
│           └── hb
│               └── hb743g47
│                   └── default-channel.txt
├── entries
│   └── 2022
│       └── 4
│           ├── 0c
│           │   └── 0c53j2ag
│           │       ├── audiences.txt
│           │       ├── channels.txt
│           │       ├── index.html
│           │       └── mf2.json
│           ├── 0h
│           │   └── 0hhgilhi
│           │       ├── audiences.txt
│           │       ├── channels.txt
│           │       ├── index.html
│           │       └── mf2.json
│           ├── 12
│           │   └── 125kgb23
│           │       ├── audiences.txt
│           │       ├── channels.txt
│           │       ├── index.html
│           │       └── mf2.json
│           ├── 19
│           │   └── 19c84e12
│           │       ├── audiences.txt
│           │       ├── channels.txt
│           │       ├── index.html
│           │       └── mf2.json
│           ├── 2m
│           │   └── 2mc11eki
│           │       ├── audiences.txt
│           │       ├── channels.txt
│           │       ├── index.html
│           │       └── mf2.json
│           ├── 56
│           │   └── 564i2f3e
│           │       ├── audiences.txt
│           │       ├── channels.txt
│           │       ├── index.html
│           │       └── mf2.json
├── misc
├── rel
└── syndication
    └── 2022
        ├── 2
        │   ├── 0d
        │   │   └── 0ddjaeic
        │   ├── 0f
        │   │   └── 0fi9273i
        └── 3
            ├── 0h
            │   ├── 0h040jgf
            │   └── 0hchfc2j
            ├── 22
            │   └── 22be5m9d

538 directories, 592 files (truncated here on the Wiki)
  • The root directory name defaults to ./data but can be put anywhere.
  • Most resources are stored at $ROOT/$RESOURCE_ROOT/$LOCAL_PATH where $LOCAL_PATH is a string with $CURRENT_YEAR/$CURRENT_MONTH/$RANDOM_ID_FIRST_TWO_CHARS/$RANDOM_ID_WHOLE and $RESOURCE_ROOT represents the root path for a resource.
  • The only 'special' resource is the database—it doesn't get its own directory.
Resource Type to Relative Directory Location
entries entries
contacts contacts
categories categories
channels channels
destinations destinations
relation links rel

Database

SQLite is used to handle quick indexing. Koype implements a virtual table that picks up JSON information for entries on disk on every row access to allow for querying for anything that I've liked from a specific URL like: 

-- Load the module only available in-process.
CREATE VIRTUAL TABLE epv USING MODULE entry_property_values();
-- Select any entries that use the provided URL.
SELECT
    uid
FROM epv, json_each(epv.properties, '$.like-of')
WHERE
    (
        -- nested item
        json_type(json_each.value) == 'object' AND
        (
          json_extract(json_each.value, '$.value') = :expected_link OR
          json_extract(json_each.value, '$.value') LIKE :expected_link
        )
    ) OR (
        -- nested image
        json_type(json_each.value) == 'object' AND
        (
            json_extract(json_each.value, '$.src') = :expected_link or
            json_extract(json_each.value, '$.alt') = :expected_link
        )
    ) OR (
        -- a string
        json_type(json_each.value) == 'text' AND
        json_each.value = :expected_link
    )
ORDER BY uid DESC LIMIT 3

IndieWeb Examples

Jacky Alciné

I use it for my own site at https://jacky.wtf. Planning to use it for my projects home page and to provide hosting via Koype Hub.

See Also

Retrieved from "https://indieweb.org/wiki/index.php?title=Koype&oldid=90408"