For a whille now, I have seen scans that contain the pattern “%%target%%” in the URL. For example, today this particular URL is popular:
For a whille now, I have seen scans that contain the pattern “%%target%%” in the URL. For example, today this particular URL is popular:
How does Reddit block users? Just by headers and IP addresses? Could the same be done once the GPTbot headers are known?
Every bit of information being sent to your web server can be spoofed. There is nothing you can do about this unless you’re willing to exclude an increasing percentage of real users.
My webserver is constantly barraged by crawlers and bots because I have zero defenses. I’ve considered intercepting the obvious ones, like the ones targeting wordpress plugins. I don’t use wordpress. I could serve them a 200 instead of a 404 and hopefully waste a real humans time if they check the hits manually.
Header spoofing is scraping 101