21 votes

DuckDuckGo now crawls the web regularly to create a free list of trackers to block

5 comments

  1. [2]
    kfwyre
    Link
    Seems like DuckDuckGo has been making a big push recently. I've seen a good amount of ads for them, all focused on privacy. One of my friends even saw an IRL billboard for them and only recognized...

    Seems like DuckDuckGo has been making a big push recently. I've seen a good amount of ads for them, all focused on privacy. One of my friends even saw an IRL billboard for them and only recognized the company because I'd talked about it. Their question: "do you still use that 'duck' search site?"

    6 votes
    1. intuxikated
      Link Parent
      Yeah I've seen some pictures of the billboards on reddit.

      Yeah I've seen some pictures of the billboards on reddit.

      3 votes
  2. intuxikated
    Link

    For instance, one of the most popular lists used by blocking software, EasyList, comprises nearly 100,000 rules — URLs or strings it instructs the blocker to watch out for. This is a great resource, but also, owing to the fact that it has been manually curated over several years, a bloated one: Thousands of those rules may no longer be accurate or relevant, and most won’t come into play during the average user’s browsing session hitting a few of the top 100 sites out there.

    DuckDuckGo’s approach is to start with a clean slate and use web crawlers — virtual online agents that visit and catalog selected aspects of sites — to build a rolling database of rules that adapts to the latest jukes by trackers and site admins.

    This database can be both comprehensive and flexible, as it compares the patterns of some 50,000 sites to find rules and associations. Tracking has become highly sophisticated, and methods exist to use multiple sites and services to identify a user who has opted out of cookies and other traditional signals. By comparing behaviors of many sites regularly and profiling the techniques employed across them, the resulting data is rich and up to date.

    1 vote
  3. [3]
    Comment deleted by author
    Link
    1. [2]
      cutchyacokov
      Link Parent
      Interesting. It got through my Pi-Hole filters and the page rendered reasonably well without javascript on my umatrix / ublock origin setup. I would have given their page 10/10 for being readable...

      Interesting. It got through my Pi-Hole filters and the page rendered reasonably well without javascript on my umatrix / ublock origin setup.

      I would have given their page 10/10 for being readable and even having a couple relevant images load through my setup. Do you know what specifically was caught on yours?

      3 votes
      1. [2]
        Comment deleted by author
        Link Parent
        1. cutchyacokov
          Link Parent
          Also interesting. I don't see advertising.com anywhere in my Pi-Hole logs so they may be using different advertising platforms depending upon location.

          Also interesting. I don't see advertising.com anywhere in my Pi-Hole logs so they may be using different advertising platforms depending upon location.

          1 vote