• 0 Posts
  • 42 Comments
Joined 1 year ago
cake
Cake day: July 22nd, 2023

help-circle









  • robots.txt does not work. I don’t think it ever has - it’s an honour system with no penalty for ignoring it.

    I have a few low traffic sites hosted at home, and when a crawler takes an interest they can totally flood my connection. I’m using cloudflare and being incredibly aggressive with my filtering but so many bots are ignoring robots.txt as well as lying about who they are with humanesque UAs that it’s having a real impact on my ability to provide the sites for humans.

    Over the past year it’s got around ten times worse. I woke up this morning to find my connection at a crawl and on checking the logs, AmazonBot has been hitting one site 12000 times an hour, and that’s one of the more well-behaved bots. But there’s thousands and thousands of them.


  • If cookie prompts annoy you (and why wouldn’t they? Complicated and time wasting prompts caused by terrible and compromised legislation that’s led to far more intrusion instead of enforcing use of browser settings) and you don’t care about cookies, then the browser extension “I don’t care about cookies” suppresses the vast majority.



  • I think this type of scheme is illegal under the GDPR, which is in effect in the UK just as it is in the EU.

    It’s been a while since I worked with the GDPR, but from memory the wording is such that:

    The data holder needs to allow people to opt out of data collection. The subject can request to be forgotten. The data holder explicitly cannot charge for this.

    But changes move slow, and The Mirror is probably banking on nobody caring enough to complain, and Trading Standards being too underfunded and swamped with other work to investigate otherwise (which they are). If they’re challenged, they’ll just change tack, go “oops” and are unlikely to hit big fines unless they dig in.

    Cookie laws are a horrible mess and always have done - the resulting consent banners are far more intrusive than anyone wanted.




  • We recently researched these for work.

    They tick a lot of boxes - lots of space, reasonable speed, great cold storage figures. Reasonably priced tapes. Agree, they’re the best thing. The slow read speed isn’t quite as bad as expected (They can go extremely fast in seek mode), but definitely something to consider. We were okay with that for our needs.

    But damn, the price of the hardware was horrendous - we got priced (I think) close to £20k for a suitable drive that met our needs. Completely killed the project. And remember that if you’re doing site replication for DR, you’ll need at least two of them. Sadly, it looks like we’ll be using external HDD’s for a while longer…