The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Tuesday, August 05, 2025

A bit of a deep dive into the Feedly bot

This Lobsters thread got me looking at Feedly again. Last month, the Feedly bot made the following requests to the Atom feed for my site:

Feedly Agents for July 2025
Resource Response Agent Requests
/index.atom 200 Feedly/1.0 2058
//index.atom 200 Feedly/1.0 (poller) 86
/index.atom 200 Feedly/1.0 (+http://www.feedly.com/fetcher.html; 37 subscribers; ) 19
/index.atom 200 Feedly/1.0 (+http://www.feedly.com/fetcher.html; 16 subscribers; ) 19
//index.atom 200 Feedly/1.0 (+http://www.feedly.com/fetcher.html; 8 subscribers; ) 19
/index.atom 200 Feedly/1.0 (poller; 37 subscribers; ) 11
/index.atom 200 Feedly/1.0 (+https://feedly.com/poller.html; 37 subscribers; ) 1
/index.atom 200 Feedly/1.0 (+https://feedly.com/poller.html; 16 subscribers; ) 1

One suggestion for one request for /index.atom having 37 subscribers, and another for /index.atom having 16 subscribers was one was originally for http: and the other for https:. That's a decent explanation, given we have 8 subscribers for //index.atom, telling me that Feedly is treating http://boston.conman.org/index.atom, https://boston.conman.org/index.atom and https://boston.conman.org//index.atom as entirely separate feeds, even though I now redirect http: to https:. But even though I do redirect http: to https:, it's with a temporary redirect, not a permanent one (because I'm still wary about making the redirect permanent) so that one is totally on me; the //index.atom is obviously a typo so that one is totally on Feedly.

I still can't tell the difference between the fetcher and the poller. Even the pages describing the two are identical, except one says “Fetcher” and the other says “Poller.” That's just really weird. And what's with the plain “Feedly/1.0” bot?

The 200 response means that Feedly did not do a conditional fetch of the feed (Feedly can ask “Did the file change since I last requested it?” and my server can reply with either “Yes, here it is” with a 200 response, or “No, it did not” with a 304 response). I did go back before I {^2022/12/04.1 switch to https:) and there, (from November of 2022) I get a completely different Feedly:

Feedly Agents for November 2022
Resource Respose Agent Requests
/index.atom 304 Feedly/1.0 (+http://www.feedly.com/fetcher.html; 27 subscribers; like FeedFetcher-Google) 2656
/index.atom 304 Feedly/1.0 (+http://www.feedly.com/fetcher.html; 28 subscribers; like FeedFetcher-Google) 477
/index.atom 200 Feedly/1.0 (+http://www.feedly.com/fetcher.html; 27 subscribers; like FeedFetcher-Google) 40
/index.atom 200 Feedly/1.0 (+http://www.feedly.com/fetcher.html; 28 subscribers; like FeedFetcher-Google) 8

Aside from picking up one more subscription from Feedly, it's what I would expect—most requests are conditional with none of that “fetcher/poller” stuff. I can explain the current lack of conditional requests on the http: to https: redirect throwing off the request code, since that seems to hold true starting with December of 2022. But in June of 2023, when I get my first https: subscriber, there is no conditional requests:

Feedly Agents for June 2023
Resource Response Agent Requests
/index.atom 200 Feedly/1.0 (+http://www.feedly.com/fetcher.html; 32 subscribers; ) 227
/index.atom 200 Feedly/1.0 (+http://www.feedly.com/fetcher.html; 1 subscribers; ) 136
/index.atom 200 Feedly/1.0 (+http://www.feedly.com/fetcher.html; 33 subscribers; ) 74

So maybe their https: request code fails to do conditional requests? Odd, but it does explain why there are no conditional requests. It also appears that they tuned their polling down from November of 2022 to June 2023. Seems like a few easy bugs to fix to me.

But then in September of 2023, the number of requests for the https: version shoots up over 4,000, almost 6,000 in October 2023, and in November of 2023 the “poller” first shows up, only to go away in December of 2023, only to show up again in March 2024 and stick around from there. So it's clear to me that the backend at Feedly changed, and from my point of view, not for the better.

Obligatory Picture

… over many a quaint and curious volume of forgotten lore …

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

Obligatory AI Disclaimer

No AI was used in the making of this site, unless otherwise noted.

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: https://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

https://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2025 by Sean Conner. All Rights Reserved.