• VonReposti@feddit.dk
      link
      fedilink
      English
      arrow-up
      14
      ·
      2 days ago

      They’re spamming all web logs too with an advertisement for their services in the user agent. I decided to ban them from all my websites because the logs took up too much space.

        • VonReposti@feddit.dk
          link
          fedilink
          English
          arrow-up
          2
          arrow-down
          1
          ·
          2 days ago

          They’re saying who they are, what they do, and are linking to their website and sometimes sends hundreds of requests in a minute. It might not say "For only €49.99 you can get your very own thing!”, but that does not mean they aren’t throwing their name up in every website owner’s arse whether they like it or not.

            • VonReposti@feddit.dk
              link
              fedilink
              English
              arrow-up
              3
              ·
              1 day ago

              That could have been done by just having a single entry called GoogleBot or BingBot, not an entire sentence explaining their product offering let alone hundreds of times a minute.