Info for Site Owners
Why Should I Add My Site to The Feed Directory?
Blogs come in all shapes and sizes from newspapers, to personal blogs to twitter-like microblogs, and The Feed Directory is the open search directory for it all. Adding your site to The Feed Directory's growing index of blogs, microblogs, and photoblogs can help your site get more exposure and help Pine.blog users more easily discover and recommend your site. Additionally, Pine.blog directory is available to other apps via the official API, so once you add your site to The Feed Directory, it will show up in any app using Pine.blog's API.
What Happens When I Add My Site?
When a user submits a link to The Feed Directory's index, a request is sent to the site and its contents are crawled to determine if it's a feed, or a site. Feeds are added directly to the index with a link to their parent site. A site, however, is inspected for any feeds on the given page and any feeds found this way are tied to the site.
Once a site has been inspected, Pine.blog needs to determine which feed (if multiple) is the site's main feed since, by default, when a user follows a site they're subscribed to its main feed. The Feed Directory will always prefer links to Feedburner and JSON Feeds when multiple feeds are found. If no feed falls into this category, then the feed with the shortest URL is used (since this is most likely to be the home feed of the site). In all cases, feeds that are most likely comments feeds (i.e. WordPress Comments Feeds) are almost never considered the main feed.
In an effort to simplify and to prevent spam, The Feed Directory populates a site's data with information found in its feeds. No attempts are made to gather information from the host site since RSS and JSON Feeds provide all the information needed.
Improving Your Search Appearance
Ranking highly in Feed Directory search results is important. Search is the most common way that users will discover your site and its feeds. The Feed Directory uses the titles and contents of recent posts, the titles and descriptions of your feed, and the categories that your feed provides as searchable items. To ensure that your site and its feeds appear in the search results properly, its important to properly implement these elements in your feed.
In some cases, The Feed Directory will also crawl pages on your site that are linked to by a given feed in order to get additional data (images, context, metadata, etc). Implementing standards like OpenGraph and having good, descriptive meta tags is critical to helping The Feed Directory properly index your site.
Logos and Images
The Feed Directory has multiple methods for detecting the proper image to use for your site and for individual posts. These two need not be the same image either. Pine.blog and The Feed Directory encourage sites to have distinct logos and to promote the individual authors of posts at the same time.
Site images are discovered in the following order:
- Images linked to in the site's default feed (i.e. RSS feed
<image>tags) - Special Cases (YouTube Users, Mastodon Users, SubReddit Logos, etc)
- Icons in your site's HTML including
rel=apple-touch-iconandrel=iconlinks.
In the cases where a site has multiple feeds, the following order is used for each feed:
- Images linked to in the feed (i.e. RSS feed
<image>tags) - The parent site's image.
Individual posts in a feed are given a few more options. This allows sites to provide images for each item in the feed, as well as a site-wide logo.
Images for posts in a feed use the following order:
- Images linked to in the entry for the given item (i.e. RSS feed
<image>tags inside the<item>for the given item). - The parent feed's image.
- The parent site's image.
Subscriber Counts
When The Feed Directory crawls a feed on your site looking for new posts, the User-Agent header in the request will have two important pieces of information that you can use for your own analytics: the id of the feed in The Feed Directory, and the number of users on Pine.blog that are subscribed to that feed. For reference, the header you'll find will look something like this:
User-Agent: Pine.blog/1.5 Feed Crawler (156 subscribers; feed-id=some-uuid; +https://pine.blog/)
Important Note: The Feed Directory also requests various resources from your site when importing a site for the first time, or when a feed changes. These requests are often made to your site's home page or to any media that's linked in new posts. Requests made this way will not include any subscriber counts. Counts are only present in requests for the actual feed.
Additionally, The Feed Directory tries to determine various metadata about a post from its contents and although The Feed Directory does make requests to your site for linked media, they are not cached or stored by The Feed Directory. Users will always request resources and media directly from your servers.
Many other feed readers and podcast clients use this method to report subscriber counts. Sites can use this code to gather information about their active subscriber counts.
Addition Crawling Info
Please ensure the following servers are not prevented from accessing your site. Blocking these servers will cause all Pine.blog users from being able to receive updates from your site.
worker1.pine.blogworker2.pine.blogworker3.pine.blogworker4.pine.blogworker5.pine.blog
Standard Caching Headers are obeyed for up to 1 day. The Feed Directory will automatically refetch the contents of your site at least once per day.
Payments and Contact Information
When Pine.blog crawls your feed, it may make additional requests to the pages linked to by the feed. When doing so, The Feed Directory collects rel=payment and rel=contact links. The Feed Directory will show icons on your site's pages and in the Pine.blog app to encourage people to contact you and support your work.
Pine.blog does not handle or process your payments in any way nor does Pine.blog handle any communications between your site and your readers. It just displays a link to any resources you've added to let your readers know how to support your work and get in touch.
Webmentions
Pine.blog sends and receives Webmentions. When a Pine.blog user likes, comments on, or replies to a post, Pine.blog will send Webmentions to the relevant site. To receive Webmentions for Pine.blog, all you need to do is ensure that your site is compatible with the Endpoint Discovery methods described in the Webmention Specification (all methods are supported).
Webmention Requests will always be made with the following User-Agent string.
Pine.blog/1.0 Webmention Discovery [email protected]
Pine.blog also receives Webmentions. You can send Webmentions to the following URL:
https://pine.blog/api/webmention
Webmention Icons and Follower Status
Pine.blog uses h-entry classes in the target page's markup to determine what icons, name, and URL to show in the content of the Webmention. Pine.blog supports replies, mentions, and likes. Reposts are shown as mentions.
Users can elect to only show the comments, mentions, or replies from sites they follow on their blog. In this case, Pine.blog uses h-entry tags (specifically the p-author and u-url tags) to determine if a given site is one that the user follows. If your site does not support h-entry classes, then it will be omitted in all cases where the user has selected this option.