user-agent

This article is a stub. You can help the IndieWeb wiki by expanding it.

User-agent is a common HTTP header that generally indicates the name, version, and a URL for the application making the request, and is also a robots_txt command (User-agent:) that tells specific robots by name to obey the following commands.

As a simple anti-spam measure, many sites (especially Wordpress?) block requests from User-Agents like curl and python-requests, and it can be useful to specify your own to avoid these restrictions.

If your application goes haywire and starts spamming them with too many requests, it also gives the receiver a clue where it's coming from so they can let you know!

IndieWeb Examples

Pelle Wessman

• Pelle Wessman's webmention endpoint "A-WebMention-Endpoint/0.8.1 (https://github.com/voxpelli/webpage-webmentions)"

Bridgy

• Bridgy uses "Bridgy https://www.brid.gy/about"

Woodwind

• Woodwind uses "Woodwind https://github.com/kylewm/woodwind"

Jeremy Keith

Jeremy Keith's minimum viable webmention code sets the User Agent as "mydomain (webmention.org)"

WordPress

WordPress, when verifying pings, uses "WordPress/4.5;http://www.example.com; verifying pingback from 0.0.0.0." which is a non-standard usage.

Silos

various Silos use specific user-agents when they crawl a page, e.g. to create URL previews. You can use this if you want to give them special treatment, e.g. only serve silo-specific tags to them, work around HTTPS issues [1],…

• Twitter's contains Twitterbot

Resources

• http://www.projecthoneypot.org/harvester_useragents.php
• http://www.askapache.com/htaccess/blocking-bad-bots-and-scrapers-with-htaccess.html

See Also

• HTTP
• Apache
• htaccess
• spam
• History of the browser user-agent string
• https://podnews.net/article/podcast-app-useragents
• Check yours (and apparent IP address) using http://whatsmyuseragent.org/