As well as the general dumbing down and relentless removal of search options, it covers the new technologies that Google is experimenting with: artificial intelligence, driver-less cars, robotics, home environment sensors and controls. Some of this is already being integrated with search and “mobile”.
I am running a “New Google, New Challenges” workshop for UKeiG this autumn in Manchester and London. It concentrates on search, how the changes at Google are impacting the way it manages our search and presents results, and how to use what is left of the advanced search techniques and specialist databases for more relevant research results.
Search engines, government and official information sources, and the EU regulatory environment are continually changing. All of these affect how we search and the information that is presented to us. In some cases information may be deliberately excluded from our results. This one day workshop will look at what’s new, key resources for business and official information, and how to use search tools to ensure you are picking up everything that you need. There will be time for practical sessions so that you can try some of the exercises provided, or experiment with your own searches. Lunch and refreshments are included.
Topics covered include:
effect of EU legislation on research and due diligence
increase in official open data – accessibility, quality, usability
changes to Google and other search tools, and their impact on research
starting points, evaluated listings and government sources
company information: official sources; free open data sources worldwide; companies that repackage official company information – pros and cons
news sources and alerting services
the value of social media and professional networks for business intelligence
Last week’s workshop on Google had a wonderful mix of participants from the academic, government, public, NHS and legal sectors. And true to form, Google decided to change a few things on the day. The link to the Google News advanced search completely disappeared (it wasn’t that good anyway!) but now seems to have reappeared. The search options that appear at the top of the results pages had changed compared with screen shots that I had taken 2 days previously; they options now seem to change according to the type of query so we suspect that this is an example of Hummingbird (Google’s new algorithm). The Google custom search engine interface has changed yet again and presented challenges to even those of us who are regular users. And then there was the new Google log-out/log-in interface which had us all flummoxed until the end of the day. (That merits a separate blog rant).
In the time honoured tradition, at the end of the day the group was asked to come up with their top 10 tips for searching Google. Here is what they came up with.
Verbatim Several people shouted out this one as the number 1 tip for searching Google. Google automatically looks for variations on your search terms and sometimes drops terms from your search without telling or asking you. To make Google run your search exactly as you have typed it in, first run your search. Then click on ‘Search tools’ in the menu above your results, in the second line of options that appears click on ‘All results’ and from the drop down menu select Verbatim.
site: command Use the site: command to focus your search on particular types of site, for example site:ac.uk, or to search inside a large rambling site. You can also use -site: to exclude sites from your search.
Advanced search commands and search options Learn how to use the search commands (for example intext:, filetype: and site:), Many of these can be used on the advanced search screen that can usually be found under the cog wheel in the upper right hand area of the screen, but that link sometimes disappears so learning the commands is a better bet. A list of the more useful Google commands is at http://www.rba.co.uk/search/SelectedGoogleCommands.shtml
intext: Google’s automatic synonym search can be helpful in looking for alternative terms but if you want a term to be included in your search exactly as you have typed it in then prefix the word with intext:.
Country versions of Google. The country versions of Google give priority to the country’s local content, although it might be in the local language. This is a useful strategy when searching for industries, companies and people that are active in a particular country. Use Google followed by the standard ISO two letter country code, for example http://www.google.de/ for Google Germany, http://www.google.no/ for Google Norway.
Google.com Apart from presenting your search results in a different order Google.com is where Google launches new features and search options first. As well as seeing pages that may not be highly ranked in Google.co.uk you will get an idea of how Google search may look in the UK version in the future. It also has some unique search options such as recipes!
Click on the link to see the full list, which might include free or open access copies of the paper.
Search by image Click on the camera icon in the image search bar to upload a photo or link to an image on the web. Google will then try and find similar images. There were comments from some of the workshop participants that this does not seem to always work as well as it used to, which reflects my own recent experience of the option. It is still worth a go, though, if you want to find different versions of an image.
Hummingbird Keep an eye out for new layouts and ways of searching that are now appearing since Hummingbird was launched. For example, you can now compare the properties of two similar items: compare cabbage with spinach will show a table comparing the nutritional value of the two vegetables.
Google Custom Search Engines (CSE) Several of the participants had a go at setting up their own CSE. Ideal for bringing together websites that you search individually on a regular basis.
Delegates at yesterday’s business information workshop in London came up with an interesting combination of websites and search commands for their end of day top tips.
OFFSTATS – Official Statistics on the Web http://www.offstats.auckland.ac.nz Excellent starting point for official statistical sources by country, region subject or a combination of categories. All of the content in the database is in the public domain and available through the Internet.
GMackerhttp://gmacker.com/web/content/gDateRange/gdr.htm Google’s Verbatim in the search options menu on the results page is great. Google’s date option from the same menu is great. But you cannot use both together. You can use the daterange: command, though, with Verbatim but it’s complicated. GMacker is a much easier way to do it. Type in your search on the GMacker page, select your dates from the calendars and click on ‘Google Search’. When the results appear on Google simply apply Verbatim in the usual way.
Zanranhttp://zanran.com/This is a search tool for searching information contained in charts, graphs and tables of data and within formatted documents such as PDFs, Excel spreadsheets and images. Enter your search terms and optionally limit your search by date and/or format type.
News alerts, news curation services and automated newsletter generation. Use Google alerts, RSS feeds and newsletter generation sites such as Paper.li (http://paper.li/) and Scoop.it (http://www.scoop.it/) to keep up to date and share news with colleagues.
site: command Use the site: command to focus your search on particular types of site, for example site:ac.uk, or to search inside a large rambling site. You can also use -site: to exclude sites from your search.
Numeric range This command is unique to Google. Use it for anything to do with numbers – years, temperatures, weights, distances, prices etc. Simply type in your two numbers separated by two full stops as part of your search. A good way of limiting your search to pages or news covering a company’s activities over two or three years in the past.
OpenCorporates “The Open Database Of The Corporate World” http://opencorporates.com/ Provides access to open corporate data on 55 million companies in 75 jurisdictions. You can search all jurisdictions at the same time or select just one. Results can be filtered by type of data held, current status, company type, SIC. A link to the original registry page for a company is always included with the displayed information.
DataMarket http://datamarket.com/ A portal to thousands of free and priced datasets. Free to search, and create charts and visualisations of the free data.
Towards the end of June I headed off to Swansea Central Library to facilitate a workshop on search tools and techniques for finding business information and statistics. The session was organised for the libraries of the wonderfully named SWAMP – South West and Mid Wales Partnership.
We had fantastic views from the library of the sea and shore line so they did very well to remain focused on the work in hand. The top tips that the group suggested at the end of the day were a mixture of search techniques and business information sites.
Don’t give up and don’t get stuck in a rut. If your first attempts fail to produce anything useful try a different approach to your search. Try some of the tips mentioned below: use advanced search commands, a different search tool or go direct to a website that covers your subject area or type of information.
Google automatically looks for variations on your search terms and sometimes drops terms from your search without telling or asking you. To beat Google into submission and make it run your search exactly as you have typed it in, click on ‘Search tools’ in the menu above your results, then click on the arrow next to ‘All results’ and from the drop down menu select Verbatim.
3. Private Browsing.
To stop search engines personalising your results according to your previous searches and browsing behaviour, find out where the private browsing option is in your browser (in Chrome it is called Incognito). This ignores all cookies and past search history and is as close as you can get to unfiltered results.
Short cuts to private browsing in the main browsers are:
Chrome – Ctrl+Shift+N
FireFox – Ctrl+Shift+P
Internet Explorer – Ctrl+Shift+P
Opera – Ctrl+Shift+N
Safari – click on Safari next to the Apple symbol in the menu bar, select Private Browsing and then click on OK.
4. The site: command.
Include the site: command in your search to focus your search on particular types of site, for example site:ac.uk, or to search inside a large rambling site. You can also use -site: to exclude sites from your search. For example, if you are searching for information on Wales and Australian websites mentioning New South Wales keep coming up include -site:au in your search.
5. The filetype: command.
Use the filetype: command to limit your research to PowerPoint for presentations, spreadsheets for data and statistics or PDF for research papers and industry/government reports. Note that in Google filetype:ppt and filetype:xls will notpick up the newer .pptx and xlsx formats so you will need to incorporate both into your strategy, for example filetype:ppt OR filetype:pptx, or run separate searches for each one. In Bing.com, though, filetype:pptx will pick up both .ppt and .pptx files.
6. Guardian Data Store (http://www.guardian.co.uk/data/)
For datasets and visualisations relating to stories in the news. This is proving to be a very popular site on both the public and in-house workshops. As well as the graphs and interactive maps the source of the data is always given and there are links to the original datasets that are used in the articles.
7.Company Check (http://www.companycheck.co.uk/)
Company Check repackages Companies House data and provides 5 years of figures and graphs for Cash at Bank, Net Worth, Total Liabilities and Total Current Liabilities free of charge. It also lists the directors of a company. Click on a director’s name and you can view other current and past directorships for that person.
10. Keep up to date Keep up to date with what the search engines are up to, changes to key resources and new sites. Identify blogs and commentators that are relevant to your research interests and subject areas and follow them using RSS or email alerts.
Picture the scene: an obviously distressed researcher is hunched over a computer screen, sobbing hysterically. All they wanted was a list of donkey sanctuaries in Surrey. How difficult is that? But Google decided that what they really wanted was a field guide to identifying buttercups. Our researcher tries all the advanced search commands and options they know but to no avail. It seems that Google has locked them into its dreaded live experiments (1) with no possibility of escape, and the information is needed NOW.
There is hope, though. There are other search engines out there. Bing may seem consumer/retail focused, but its list of advanced search commands is great at unearthing serious research information that Google buries at around the 2 millionth entry in your results list. My comparison and summary of search commands at http://www.rba.co.uk/search/compare.shtml lists the Bing commands that you are most likely to need. Or if you just want a no nonsense summary of your topic without all of Google’s personalisation and experiments look no further than DuckDuckGo. But should you even be using Google or similar, generic search engines in the first place? Think about the type of information you are looking for.
The advanced Google workshop that I am running for UKeiG (How to make Google behave) has a new venue. It is still being held in Manchester but will now be in the 4th Floor Teaching Suite, Main Library, University of Manchester M13 9PP. The date remains unchanged (April 30th, 2013).
We shall be looking at what goes on “underneath the bonnet” and covering Google’s advanced commands and search options in detail. We’ll also be reviewing Google’s specialist tools including the Public Data Explorer, Scholar and many more. As usual with my workshops there will be time allocated for practical sessions so that you can try out the techniques for yourself. Further details and booking information are available on the UKeiG website at http://www.ukeig.org.uk/trainingevent/make-google-behave-techniques-better-results-karen-blakeman
Anyone who has attended one of my workshops knows that I ask the group to propose at the end of the session their top tips. These are the Canterbury group’s top 10 tips.
1. What’s going on?
Try and find out what’s going on behind the scenes and how the different search tools work. For example, Google and Google Scholar are quite different in the way they manage your search. Understanding how they operate means that you can adapt your search strategy accordingly and also manage your expectations; for example Google Scholar does not use the publishers’ meta data so author and date search are unreliable.
2. Personalisation and ‘unpersonalisation’
If you use Chrome as your browser, open what is called an incognito window. In the top right hand corner of your screen there is an icon with three lines. Click on it and from the drop down menu select New incognito window. Alternatively press the Ctrl Shift N keys on your keyboard
If you use Firefox, from the menu at the top of the screen select Tools followed by Start Private Browsing.
In Internet Explorer select Tools followed by InPrivate Browsing. If you cannot see InPrivate under Tools try looking under the Safety option.
3. Advanced search commands
Use Google advanced commands such as filetype: to focus on PDFs, presentations, spreadsheets containing data and site: to look for information on just one site or a range of sites such as UK government. Although the advanced search screen has boxes for you to fill in for the commands the file format or filetype option is limited. It does not include options for the newer Microsoft Office formats such as .pptx and xlsx. Use filetype: as part of your search strategy, for example:
nasa dark energy dark matter filetype:pptx
Google Scholar commands are more limited – see slide 28 of the presentation.
Google automatically looks for variations on your terms and sometimes omits words from your search if it thinks the number of results is too low. Prefixing a term with intext: tells Google that it must be included in your search and exactly as you have typed it in. For example:
UK public transport intext:biodiesel statistics
tells Google that biodiesel must be included in the search and exactly as typed in.
5. Reading Level
Use Reading level if Google is failing to return any research oriented documents for a query. Run the search and from the menu above the results select Search tools, All results and then from the drop menu Reading level. Options for switching between basic, intermediate and advanced reading levels should then appear just above the results. Google does not give much away as to how it calculates the reading level and it has nothing to do with the reading age that publishers assign to publications. It seems to involve an analysis of sentence structure, the length of sentences, the length of the document and whether scientific or industry specific terminology appears in the page.
6. Date options
In Google web search, use the date options in the menus at the top of the results page to restrict your results to information that has been published within the last hour, day, week, month, year or your own date range. Click on Search tools, then Any time and select an option. This works best with news, discussion boards, and blogs and web sites that use blogging software to generate pages but Google is getting better at identifying the correct date of a web page.
Google Scholar handles publication dates differently. On the results page you can select a date range from the menu on the left hand of the page. Alternatively, you can run a Google advanced search and enter your publication years. However, Google Scholar looks for publication years in the area of the document where the date is most likely to be. As a result it may identify a page number or part of an author’s address as a year!
7. Google Scholar alerts
To be used with caution as the searches periodically stop without warning, and so have to be set up again, and they sometimes include documents that are several years old. Whatever your search you can set up an alert by selecting Create alert from the menu on the left hand side of the results page.
If the author has created a profile on Google Scholar, from their profile page you can follow new articles and/or new citations for that author. From past experience I warn you that this is not entirely reliable.
8. Metrics – top publications Although it claims to search all scholarly literature Google Scholar does not always cover all of the key journals in a subject area. There is no complete source list but there is a top publications for subjects and languages under the ‘Metrics’ link in the upper right hand corner of the Scholar home page.
9. Microsoft Academic Search – visualisations
Microsoft Academic Search (http://academic.research.microsoft.com/) is a direct competitor to Google Scholar. The site is sometimes slow to load and it often assigns authors to the wrong institution. Nevertheless, the visualisations such as the co-author and citation maps can be useful in identifying who else is working in a particular area of research. The visualisations can be accessed by clicking on the Citation Graph image to the left of the search results or author profile.
10. Mednar visual Deep Web Technologies has developed in conjunction with various institutions a number of science and research specific portals, some of which are publicly available. The sources that they cover are different but they all have similar search and display options. Results are automatically ranked by relevance but this can be changed to date, title or author. In addition to the standard relevance ranked list of results the portals create clusters of topics on the left hand side of the screen. The topics include broad subject headings, authors, publications, publishers, and year of publication and are a useful tool for narrowing down a search. Some of the portals, such as Mednar (http://mednar.com/), offer a clickable ‘visual’ of topics and sub-topics.
I am giving a talk at the first NetIKX seminar of 2013 on January 28th. The subject is “Digital native or digital immigrant – does it matter?” and looks at the issues we all face in a technology-driven world. My part of the seminar – Born digital: time for a rethink – questions the usefulness of the labels and suggests that it diverts attention away from the real issues. Graham Coult, Editor-in-Chief, Managing Information will be giving an overview of research that has been undertaken into research behaviours. Has any evidence of differences been found? What are the challenges people are facing in accessing information, what are the key factors which form these challenges and what are the implications for Information and Knowledge Management?
As usual with NetIKX events delegates will be invited to share their own experiences and observations and there will be syndicate sessions to discuss the issues in more depth.
Zanranhttp://zanran.com/ A search tool for identifying charts, graphs and tables of data within formatted documents such as PDFs, Excel spreadsheets and images. Enter your search terms and optionally limit your search by date and/or format type. Zanran comes up with a list of documents that match your criteria with thumbnails to the left of each entry. Hover over the thumbnail to see a preview of the page containing your data and further information on the document. Very useful if you are looking for industry statistics.
University library subjects guides. If you are looking for some good starting points on a subject seek out some university library subject guides. These list resources that are only available to their own students and staff but may also include links to relevant publicly accessible resources that have been assessed for quality.
Socialmentionhttp://socialmention.com/ Several social media search tools were covered in the workshop but this one received a special mention as a good general all round social media tool. It covers images, blogs, Twitter, Facebook, audi0 and bookmarks. If you are monitoring a topic you can set up email and RSS alerts.
Companies Househttp://www.companieshouse.gov.uk/ The official registry for UK companies. Other services such as Company Check (http://companycheck.co.uk/) and DUEDIL (http://www.duedil.com/), which repackage Companies House data, may provide more information free of charge but it is always worth double checking with Companies House to see if there is more up to date information and to get a full of list of the documents that are available on a company. The history and list of documents that can be ordered for a company is informative in itself. On the Companies House web site use the Find Company Information to locate the company in the register and then click on “Order information for this company”. You will then see a list of available documents. Titles such as “Struck off and dissolved” and “Application for administrative restoration” would suggest that perhaps you ought to investigate further before doing business with the company.
LinkedIn groups A couple of the workshop participants regularly use LinkedIn groups for research questions. Look for groups set up by professional and official bodies relevant to your subject.
Twitter If you are looking for a professional, research or trade association that may be able to help with your research you only need to find just one organisation on Twitter covering your topic. Then, to find others that might be useful, see who that organisation is following.
Millionshorthttp://millionshort.com/. If you are fed up with seeing the same results from Google again and again give Million Short a try. Million Short runs your search and then removes the most popular web sites from the results. Originally, as its name suggests, it removed the top 1 million but the default has changed to the top 10,000. The principle remains the same, though. Exclude the more popular sites and you could uncover a real gem. The page that best answers your question might not be well optimised for search engines or might cover a topic that is so “niche” that it never makes it into the top results.
Biznarhttp://www.biznar.com/ Biznar is a federated search engine that runs your search in real-time in about 70 resources. There is a list on the Advanced Search screen where you can deselect individual or groups of resources. The results are combined into a single list and organised on the left hand side of the screen into folders such as Topics, Authors, Publications, Publishers and Dates. These are computer generated but can help you narrow down your search. A bit erratic at times and sometimes comes up with odd results but people still thought it was worth including in the Top Tips list.
DUEDILhttp://www.duedil.com/. This service repackages Companies House data and provides some of it free of charge. The feature that won DUEDIL a place in the Top Tops is the “Group” visualisation that illustrates the connections between the company you are researching, its parent companies and subsidiaries. You have to create an account (free at the moment) to access all of the information.
SCoRehttp://www.score.ac.uk A catalogue of current and historical printed company reports held in UK libraries. The catalogue does not provide links to digitised documents but is a very quick and easy way of identifying libraries that hold hard copy reports. The participating libraries include London Business School, the British Library, Manchester Business School, City Business Library, Guildhall Library, Strathclyde University and the University of Warwick. A full list is available at http://www.score.ac.uk/collections.asp.
News and comments on search tools and electronic resources for research