Searching Twitter

Embarrassed by some of your first tweets from 2007? Wish you hadn't got involved in that virtual brawl on Twitter last Christmas? There was a time when you could safely assume that your trivial ramblings would be lost in the mists of Twitter's archive never to be seen again. A search on Twitter would only give the last few days worth of postings and Google no longer archives the whole of Twitter. Bing has a Social search at http://www.bing.com/social, which supposedly searches all of Twitter, but it does not work outside of the US. True, the Library of Congress does keep copies of every single tweet for posterity but access is only allowed for serious research purposes. So far, the Library has received about 400 inquiries but has not yet been able to provide access (http://blogs.loc.gov/loc/2013/01/update-on-the-twitter-archive-at-the-library-of-congress/). So you can breathe easily again? Unfortunately not.

There are commercial organisations such as Datasift (http://datasift.com/) and Gnip (http://gnip.com/) that analyse tweets and other social media comments, but the cost puts their services out of the reach of the casual searcher. You may find, though, that your somewhat forthright hashtagged tweets at a conference have been recorded for posterity, free of charge, by a fellow delegate (Sharing (or Over-Sharing?) at #ILI2012 http://ukwebfocus.wordpress.com/2012/11/02/sharing-or-over-sharing-at-ili2012/). And Twitter, itself, is at last providing access to historical tweets.

Download your Twitter archive

You can now download your entire collection of tweets. Note that this is only for your own tweets and not everything that appears in your Twitter stream. Go to your Twitter home page, click on the cog wheel in the upper right hand corner and select settings (figure1).

Figure1: finding your Twitter settings

At the bottom of the Settings page should be a link to request your archive. A few minutes later you should receive an email with a download link. The file is zipped and once you have unpacked it you can browse your tweets by year and month (figure 2), or search using keywords or hashtags (figure 3).

Figure 2: Browse downloaded Twitter archive by year and month

 

Figure 3: Search on #ili2012 in a downloaded Twitter archive

It is not clear, though, how often you are allowed to download your archive and there does not seem to be a top-up option.

Searching past tweets on Twitter

Twitter now also allows you to run searches on its entire archive. There is no obvious date option at the moment, not even under advanced search, so it is appears to be all or nothing, and it does not give you everything straightaway. You are first of all given what Twitter thinks are the Top Tweets for your search. If you scroll down to the bottom of the results you will see a message saying "You've reached the end of the Top Tweets for ...." with a link to "View all tweets". Twitter then loads the remaining tweets as you continue to scroll down.

About Top Tweets Twitter says:

"We've built an algorithm that finds the Tweets that have caught the attention of other users. Top Tweets will refresh automatically and are surfaced for popularly-retweeted subjects based on this algorithm. We do not hand-select Top Tweets."

There are also links at the top of the results page that enable you to view Top, All, and tweets from just 'People you follow' (figure 4).

Figure 4: Twitter search results – links to Top/All/People you follow

There are advanced search commands that can be used to include a date range in your search making it easier to search for tweets on past conferences (see https://support.twitter.com/articles/71577 for details).

For example, for tweets posted during October 2009 about the Internet Librarian International Conference:

  #ili2009 since:2009-10-01 until:2009-10-31

I am not convinced, though, that Twitter is giving me everything, even when I choose 'All'. If you really want to be sure of having continued access to past musings on an event do not abandon your own archiving strategies just yet.

Twitter's search operators

Twitter's advanced search screen is at https://twitter.com/search-advanced but interestingly there is no date option. To get the best out of Twitter search use the search operators.

Oxford University for both Oxford and University (default)
“Oxford University” for the exact phrase “Oxford University”
beer OR ale finds tweets containing either beer or ale, or both (note that OR must be in capital letters)
beer –skittles contains beer but not skittles
#ili2009 contains the hashtag #ili2009
#ili2009 filter:links contains the hashtag #ili2009 and URLs. An excellent way of finding links to conference presentations and related material.
from:karenblakeman Tweets sent by karenblakeman
to:karenblakeman Tweets sent to karenblakeman
@karenblakeman Tweets referring to karenblakeman
#ili2009 since:2009-10-01 contains #ili2009 and sent since 1st October 2009. The format is YYYY-MM-DD
#ili2009 until:2009-10-31 Contains #ili2009 and sent up to 31 st October 2009. The format is YYYY-MM-DD
snow :( contains snow and a negative sentiment
snow :) contains snow and a positive sentiment
Search this site

PDF Version