I have Reached the Limits of My Endurance with Web-Crawling Robots. The Time to Take the Nuclear Option has Finally Arrived.

submitted by GB43 from self.Internet

https://bluedwarf.top/cackle/view-post.php?post_num=3097

"The people behind web-crawling robots have recently switched tactics. They must have realized that website admins [like me] are seeing through their masquerading as Mozilla web browsers to get around blocks, so they are now using groups of 5 to 20 or more IP addresses per crawl episode. I can't continue identifying and blocking 50 or more IP addresses every day [from my small websites]. I just don't have the time, and the CPU power required to block individual IP addresses instead of blocks of IP addresses is also increasing. So, in a day or two, I will begin blocking ALL of the IP addresses of the worst hosting services that are supporting these robots. Approximately in order of egregiousness, they are: Chinanet, China Unicom, AWS, LeaseWeb, datacamp, DigitalOcean, Cloudvider, Packethub, and M247. That amounts to millions of additional IP addresses. I don't know if Chinanet and China Unicom also provide IP services for individuals. I suspect they do, but at this point, I have no other option but to block them.

I don't know why you would, but if you are visiting cheapskatesguide or Blue Dwarf from one of the above hosting services, you will be blocked. But, you can always get through from a Tor exit node. Unfortunately, Tor exits node are blocked in China, so you will have to use a bridge."

all 11 comments
sorted by:
new (suggested)
topinsightfulfun

[–]Noah8293 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

Blockchain technology is a decentralized and distributed ledger system that underpins cryptocurrencies like Bitcoin and Ethereum. It allows https://casinoutansvensklicens.ltd/ digital information to be recorded and distributed but not edited or deleted, enhancing transparency and security.

[–]magnora7 3 insightful - 1 fun3 insightful - 0 fun4 insightful - 1 fun -  (8 children)

The bots on saidit can spoof any IP address. They change IP addresses with every pageload request. They change usernames with every submission. It's very hard to stop in an automated way, which is of course their intention. I agree the problem is very out of control, and it makes cloudflare a necessity rather than an option. It's damaging free speech and damaging the town squares of the internet that real people are trying to use to genuinely communicate about important things.

[–]binaryblob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (7 children)

The bots on saidit can spoof any IP address.

The only way to do this, is if you dig a hole in the ground somewhere and inject traffic of your own. I highly doubt this is the case.

[–]magnora7 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (6 children)

The IP addresses are way outside of any "range" that I could block off, that's for sure. They can basically take on any IP address they want.

[–]binaryblob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (5 children)

Having a large collection of IP addresses in many ranges or having the capability to inject arbitrary traffic into networks (based on your "spoof any IP address") is very much not the same.

If you kept statistics of the IP addresses, you could send those to whatever government you have and if you could show that the networks themselves have been compromised, that would be a matter of national security.

If (D)DoS is a problem, you should perhaps just not host it on the WWW in the first place. The WWW is designed for a friendly academic environment, not for the hybrid warfare environment in which you are. If you want to stay on the WWW, you would have to throw just about every technique in existence at it, like Facebook does, but it's probably more expensive than you can handle.

[–]magnora7 2 insightful - 1 fun2 insightful - 0 fun3 insightful - 1 fun -  (4 children)

If you kept statistics of the IP addresses, you could send those to whatever government you have and if you could show that the networks themselves have been compromised, that would be a matter of national security.

They do not care about a site like saidit, get real. There's also a high likelihood they're supportive of the attacks anyway. Also the IPs don't fall in ranges, as I said. I've gathered long-term statistics about the IP grouping and there is no pattern!

[–]binaryblob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (3 children)

Your analysis clearly is of low quality. They wouldn't be interested in such details to save saidit, but they would be interested in knowing an adversary is on their networks, but like I said, the most likely explanation is that your analysis is flawed. If I had such a capability, I would deploy it against high value targets.

[–]magnora7 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (2 children)

They already know what's going on on their networks, they have access to the backbones. You don't understand how this works

[–]binaryblob 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (1 child)

I feel like I am talking to a primate. Feel free to tell me about your academic degrees and I might want to take you serious, even though you are not replying in any intelligent fashion; if you were an AI, I would press the thumb down button.

[–]stupidfucker 1 insightful - 1 fun1 insightful - 0 fun2 insightful - 1 fun -  (0 children)

I feel like I am talking to a primate

PRETTY MUCH EVERY THING HE SAYS ABOUT IP ADDRESSES IS INCORRECT.

https://youtu.be/RRAWb_qsRS8