Just here for good conversation with good people.

  • 0 Posts
  • 29 Comments
Joined 1 year ago
cake
Cake day: July 20th, 2023

help-circle


  • I don’t think it’s a bad idea but it’s largely dependent on the crawler. I can’t speak for AI based crawlers, but typical scraping targets specific elements on a page or grabbing the whole page and parsing it for what you’re looking for. In both instances, your content is already scrapped and added to the pile. Overall, I have to wonder how long “poisoning the water well” is going to work. You can take me with a grain of salt, though; I work on detecting bots for a living.


  • Please forgive me because after a really shitty day at work I’m going to be direct and say “it’s not you.” I don’t know you or your work situation, but I have worked with so many people that felt this way. I feel this way sometimes too. I think a lot of people on Lemmy will say they feel this way at times. Hell I bet your coworkers feel this sometimes too.

    Work sucks. It sucks more than ever too. Most companies are running on skeleton crews. Someone leaves, takes a sick day, or takes PTO, and the whole thing falls apart. You’re still expected to somehow optimize through all of this instead of the company hiring additional people.

    Fuck that. You’re awesome because you show up and do your job through all the bullshit. I have worked with people that don’t always put up the best “metrics”, or whatever bullshit measurement your middle manager uses to make themselves look good, but those folks brought other soft skills to the team that made the day better.

    You’re human. Vent. Rant. Be upset. Feel your feelings. We all know the current situation sucks and no one can blame you for needing to get it off your chest. More people need to speak up and point out how shitty it is because it’s just getting worse.

    Anyway, if you want someone to vent with then DM me and we can complain together, maybe start a riot or something.















  • Apologies for the late response!

    I’ll echo similar thoughts to what I said in another comment. Librewolf, Mullvad, and other privacy based browsers are going to be a double-edged sword. You can take me with a grain of salt but these types of browsers actually do make you stand out in terms of fingerprinting. They have their own unique signatures, and the more you tweak the more you stand out too. Does it protect your privacy? It’s really hard to tell, there’s no data to suggest one way or another that I’m aware of. But, these changes are going to make you more likely to be challenged by captcha and blocked by sites in general.

    I wish we didn’t have to try and solve this type of problem. Privacy should be a right.


  • Thanks for sharing that!

    Truthfully, Firefox is fairly easy to detect. Several facets of the API it uses makes for quick identification. For example, Firefox should be able to report its build ID. Also, it won’t report specifics about the WebGL renderer you’re using like the vendor and architecture.

    The link you shared is great and really highlights something I was thinking about today regarding this subject. The more you harden and change things the more you stand out. You’re also more likely to trigger bot detection when you alter specifics about your browser like the major version you’re on. I’ve seen some extensions change the user agent to much older major versions like Firefox 60. That’s a big red flag.

    The user agent thing was bizarre, especially since it was also on Minecraft.net! I swapped to a generic Chrome on Windows agent and it instantly started working again and let me use the site as normal again.

    Yes that is bizarre 😂 It’s not clear to me if Microsoft is using their own anti-bot solution or a third party one, but it doesn’t sound really successful with the way it’s reacting.

    Overall, I can’t help but thinking the best route is to use the same thing as everyone else but roll your own VPN and change MAC addresses. Ideally, we would have some laws against all of this but I don’t foresee that anytime soon.

    I wish I could do more to help. I’m happy to answer questions you might have, though.


  • I for one want to offer a heartfelt apology. As someone that works in this space, bots are becoming more and more sophisticated. I can’t speak for Cloudflare, but we’re definitely not interested in your personal information. As someone who also prefers their privacy on the web, the fact that bot signatures overlap with privacy-centric signatures sucks. I myself have experienced it on my mobile device with Ghostery. It’s frustrating, I know.

    Would you mind sharing the guide you used for hardening your Firefox? I’m curious to see what could potentially be triggering the issue.

    Also, I just want to say, I think it’s hilarious that a site blocked you but then allows you to continue browsing after changing your user agent. That right there is bot behavior.

    To circle back around to the actual block, I bet changing your skin executes JavaScript which flags something from the anti-bot software.