Thursday, August 21, 2025
“Bro, ban me at the IP level if you don't like me!”
More and more I think I'm coming around to Jihad Alex Schroeder's Butlerian Jihad. For reasons, I'm looking into web activity and so far, the top webbot this month is one identifying itself as “Thinkbot,” which may be related to this AI company but I can't be sure. Here's how it itentifies itself: “Mozilla/5.0 (compatible; Thinkbot/0.5.8; +In_the_test_phase,_if_the_Thinkbot_brings_you_trouble,_please_block_its_IP_address._Thank_you.)”.
Seriously,
that's it.
No URL to read up on it.
It doesn't look at the robots.txt file.
Just “bro,
ban me at the IP level if you don't like me!”
Yeah, block its IP address. You mean the 74 unique addresses it used this month alone? Checking each IP address for the ASN it's from shows the 74 address coming from 41 (41!) network blocks!
A further check showed that all the network blocks are owned by one organization—Tencent. I'm seriously thinking that the CCP encourage this with maybe the hope of externalizing the cost of the Great Firewall to the rest of the world. If China scrapes content, that's fine as far as the CCP goes; If it's blocked, that's fine by the CCP too (I say, as I adjust my tin foil hat).
In any case, I added the following network blocks to my “badbots firewall rule set:”
43.130.0.0/18 43.130.64.0/18 43.130.128.0/19 43.130.160.0/19 43.131.0.0/18 43.132.192.0/18 43.133.64.0/19 43.134.128.0/18 43.135.0.0/18 43.135.64.0/18 43.135.192.0/19 43.153.0.0/18 43.153.192.0/18 43.154.64.0/18 43.154.128.0/18 43.154.192.0/18 43.155.0.0/18 43.155.128.0/18 43.156.192.0/18 43.157.0.0/18 43.157.64.0/18 43.157.128.0/18 43.159.128.0/19 43.163.64.0/18 43.164.192.0/18 43.165.128.0/18 43.166.128.0/18 43.166.224.0/19 49.51.132.0/23 49.51.140.0/23 49.51.166.0/23 101.32.0.0/20 101.32.48.0/20 101.33.64.0/19 119.28.64.0/19 119.28.128.0/20 129.226.160.0/19 150.109.32.0/19 150.109.96.0/19 170.106.32.0/19 170.106.176.0/20
The above list probably doesn't exhaustively enummerate Tencent's network block ownership, but it's a start. The above covers 476,590 unique IP addresses (excluding the base network and broadcast address for each network block). I think it's bad that I had to do this, but with the current landscape of the Internet, it seems inevitable. We can't have nice things it seems.
