With recent headlines that the NSA has been profiling people’s online movements (the irony of saying this online isn’t lost on me), it’s very interesting to see how it has affected search engines. As privacy concerns have arisen from the situation with PRISM, Google, Yahoo and Bing have seen a small subset of the market diverge to other search engines with stricter privacy policies. One would expect that the media outrage would translate to a broader shift away from the search giants. Perhaps educating people about alternative search engines is just beginning though, as momentum seems to be picking up daily.
As more and more news stories disclose how Google tries to integrate the unique information it keeps on every searcher into their search results, and tailor these to be more and more personal, many more people are looking for sites that keep searches completely anonymous. Two such sites come to mind. Philadelphia based DuckDuckGo.com and the European based StartPage.com both proclaim to be search engines that respect privacy. Each have around 3 million searches per day.
DuckDuckGo in particular has seen its traffic rise by 2 million searches per day in the last two weeks. The website’s privacy policy states that it will not retain any personal information such as IP address, or share a user’s search history to other sites, has generated just under 3.1 million direct searches per day from June 1 to the 17. This is compared to the 1.8 million per day for May. The search engine ed: “It took 1445 days to get 1 million searches, 483 days to get 2 million searches, and then just 8 days to pass 3M searches.” The growth was detailed on DuckDuckGo’s public traffic page, which illustrates a search growth spike since the NSA story first broke in early June. The story, which claimed the government organization had acquired direct access to Google, Yahoo, Facebook, Microsoft, and other tech companies within the US, has been a lightning rod of debate within the political arena.
This growth has also been due to some big time publicity from media outlets like CNN, FOX, Bloomberg TV, and select CBS affiliates. But really, that’s just a drop in the search bucket, and it poses no threat at all to mighty Google and Bing. Even with 90 million searches per month worldwide, DuckDuckGo still only has a third of the search traffic of AOL’s US search traffic. So the privacy concerns have only moved a very small percentage of searchers away from Google. Perhaps as more news comes out about privacy concerns, this will inch up. In a 2012 survey by Pew International and the American Life Project 2000 US adults were asked whether they felt search engines should use personal data to provide better search results. There was an article written about this subject in Search Engine Land, and one chart shows the findings here:
(photo source: searchengineland.com)
65% of the people surveyed did not want personalized search results, however Yahoo, Bing and Google all show personalized search results as the default, and it is downright difficult to turn off that feature.
Gabriel Weinberg, DuckDuckGo’s chief exec, has publicly stated that DuckDuckGo isn’t useful to surveillance organizations like the NSA because it does not store any information about users, who can remain truly anonymous as they search. In comparison, other search engines like Google keep track of the IP address and other social connections about each person and stores them for 18 months, so that when two people search the same thing on Google, even when they are searching anonymously, they may be shown two completely different sets of search results based on what Google has stored as their preferences.
Preferences are based on the user’s search history. Weinberg, who funded and ran the startup DuckDuckGo engine for the first three years of its inception purposefully decided to remove all personal data from its results because he never wanted to have to turn over information to governments or for court cases, and he thought it was “creepy” that search engines kept track of everything a person searched for (such as financial information, health information and other personal queries) in order to build a filter for future searches. He also wanted to build a search engine that used the best results on the web to compile each search result, and to focus the result to answer a question instantly, so that in most cases, no click into a website is necessary to get the information being searched.
The question is, if more and more people jump from Google’s search to DuckDuckGo, is the search experience there as good, without the personal profiling? Weinberg thinks it is better. He feels that what is searched is a phrase, not specific to personalization. He also thinks that most searches can actually be answered without the user having to visit a website, and he created the instant answers section of DuckDuckGo to provide these as quickly as possible. He is about ready to launch a mobile app to make these instant answers even more prevalent on mobile devices because he feels it is a pain to wade through search results on a smart phone. There are many “instant search categories available on DuckDuckGo, and results are excellent:
DuckDuckGo is also not letting this opportunity to market their brand pass them by, as the company has been quick to reinforce their message with billboards promoting its motto: “Google Tracks You, We Don’t”. One thing is certain, they’re not pulling any punches.
(photo source: DuckDuckGo.com)
While DuckDuckGo is still a small player in the land of search engines, (comparatively, Google pulled in 13.4 billion searches for May, while Bing and Yahoo hit with 3.5 billion and 2.4 billion respectively according to ComScore) the company is still making headlines and is certainly growing.
So how does DuckDuckGo get the data for its search results? Search results on DuckDuckGo are created from data gathered by the duckduckgo bot, and hundreds of other sources. DuckDuckGo says on its Support Center page that data for search results comes from “crowd-sourced sites (like Wikipedia, which are stored in our own index), Yahoo! (through BOSS), Yandex, WolframAlpha, and Bing. For any given search, there is usually a vertical search engine out there that does a better job at answering it than a general search engine. Our long-term goal is to get you information from that best source, ideally in instant answer form.”
So for a website to rank on DuckDuckGo, it’s best not to block their bot. The same “great content” advice that the other search engines suggest works here too, as DuckDuckGo seeks to answer questions instantly, so having FAQ pages and other information that users may want to know about products and services will certainly help. Mentions from social sites, informative sites, and trusted sites will also help in a site’s ranking because DuckDuckGo’s rankings are a hybrid of 100’s of APIs from other websites. Weinburg says “We basically use the ranking of the underlying APIs, and then re-rank, omit, merge etc. on top of that–and the argument is that it comes out better on the other end.” So if a site ranks well on other search engines, it will most likely rank well on DuckDuckGo too.
Two things struck me as I did the research on how to rank for DuckDuckGo. First, the results that the search engine delivers are excellent (when you don’t want personal or local results) and unbiased. Second, Google and other search engines have been more and more frequently asked to disclose information to government agencies and lawyers. The Google transparency report shows a big increase over the last few years. Google has a lot of data about you and it must disclose this data if requested by authorities.
(Source: Google Transparency Report)
However when I want to know the best pizza place or car dealer in my area, the local results that Google and Bing shows are superior in my opinion because they show the reviews, location and an image in an easily scrolled list. Is this trade off worthwhile? Google now knows I was hungry for pizza when I was on my way to buy a new car or service my car. I’ll probably be shown numerous car ads as I search the internet for the next 30 days or so. This is a simple example, but I can still get the information I need from DuckDuckGo by modifying my search and adding in a nearby town. I think that trade off is worth it. I’ll be using DuckDuckGo much more often in the future.
Keep your eye on DuckDuckGo.com in the next few weeks. Maybe the trend will continue and there will finally be somewhat of a challenge to the 800 pound gorilla named Google.
image credit: Shutterstock