Search engines, government and official information sources, and the EU regulatory environment are continually changing. All of these affect how we search and the information that is presented to us. In some cases information may be deliberately excluded from our results. This one day workshop will look at what’s new, key resources for business and official information, and how to use search tools to ensure you are picking up everything that you need. There will be time for practical sessions so that you can try some of the exercises provided, or experiment with your own searches. Lunch and refreshments are included.
Topics covered include:
effect of EU legislation on research and due diligence
increase in official open data – accessibility, quality, usability
changes to Google and other search tools, and their impact on research
starting points, evaluated listings and government sources
company information: official sources; free open data sources worldwide; companies that repackage official company information – pros and cons
news sources and alerting services
the value of social media and professional networks for business intelligence
Documents may no longer be directly accessible from the new departmental home pages so a different approach is needed if you are conducting in-depth research. GOV.UK is fine for finding out how to renew your car tax or book your driving theory test – two of the most popular searches at the moment – but its search engine is woefully inadequate when it comes to locating detailed technical reports or background papers. Using Google’s or Bing’s site command to search GOV.UK is the only way to track them down quickly, for example biofuels public transport site:www.gov.uk. Note that you need to include the ‘www’ in the site command as site:gov.uk would also pick up articles published on local government websites. This assumes, though, that the document you are seeking has been transferred over to GOV.UK.
There have been complaints from researchers, including myself, that an increasing number of valuable documents and research papers have gone AWOL as more departments and agencies are assimilated Borg-like by GOV.UK. Some of the older material has been moved to the UK Government Web Archive at http://www.nationalarchives.gov.uk/webarchive/.
This offers you various options including an A-Z of topics and departments and a search by keyword, category or website. The latter is slow and clunky with a tendency to keel over when presented with complex queries. I have spent hours attempting to refine my search and wading through page after page of results only to find that the article I need is not there, nor anywhere else, which is an experience several of my colleagues have had. This has led to conspiracy theories suggesting that the move to GOV.UK has provided a golden opportunity to “lose” documents.
I am reminded of a scene from Yes Minister:
James Hacker: [reads memo] This file contains the complete set of papers, except for a number of secret documents, a few others which are part of still active files, some correspondence lost in the floods of 1967…
James Hacker: Was 1967 a particularly bad winter?
Sir Humphrey Appleby: No, a marvellous winter. We lost no end of embarrassing files.
James Hacker: [reads] Some records which went astray in the move to London and others when the War Office was incorporated in the Ministry of Defence, and the normal withdrawal of papers whose publication could give grounds for an action for libel or breach of confidence or cause embarrassment to friendly governments.
James Hacker: That’s pretty comprehensive. How many does that normally leave for them to look at?
James Hacker: How many does it actually leave? About a hundred?… Fifty?… Ten?… Five?… Four?… Three?… Two?… One?… *Zero?*
On one of my recent workshops I was asked if I used Google as my default search tool, especially when conducting business research. The short answer is “It depends”. The long answer is that it depends on the topic and type of information I am looking for. Yes, I do use Google a lot but if I need to make sure that I have covered as many sources as possible I also use Google alternatives such as Bing, Millionshort, Blekko etc. On the other hand and depending on the type of information I require I may ignore Google and its ilk altogether and go straight to one or more of the specialist websites and databases.
Here are just a few of the free and pay-per-view resources that I use.
There are at least a dozen statistics sites that I use on a regular basis but if I’m unsure of where to look or want to make sure I haven’t missed anything I use OFFSTATS – Official Statistics on the Web at http://www.offstats.auckland.ac.nz. A great starting point for official statistical sources by country, region subject or a combination of categories. All of the content in the database is in the public domain and available through the Internet and has been quality assessed by staff at The University of Auckland Library.
Official company information
If I want to confirm the existence of a company or obtain filings and accounts I usually go direct to the relevant official company registry. I have a list of the registries that can be searched online at http://www.rba.co.uk/sources/registers.htm. As many of my enquiries are for UK companies I am a frequent visitor to Companies House at http://www.companieshouse.gov.uk/. Some information is free but filings and accounts are priced. There are several companies that repackage Companies House data and sometimes make extra data or analysis free of charge for example Company Check at http://www.companycheck.co.uk/, which enables you to search by company or director’s name. Risk reports, information on CCJs, and some official filings are priced if you do not have a subscription to the full service.
Share price information
For free share price information I use Yahoo Finance (http://uk.finance.yahoo.com/) and Google Finance (https://www.google.co.uk/finance). Both of these services provide charts and news on shares on the major stock markets. Google’s graphs are ‘annotated’ with labels that link to news articles listed to the right of the graph, so you can see whether or not a particular event or announcement has affected a share price. Both offer free, daily historical share prices in figures. As well viewing historical graphs for share prices you can download the data as a spreadsheet.
For news alerts I use a mixture of bookmarked searches, Google email and RSS alerts, and RSS feeds from a wide range of blogs and news sites. I find that Google alerts are erratic and unreliable but they do sometimes pick up something unique so I still include them in the mix. RSS feeds are my main source of current awareness and when a news feed is rather broad in coverage I use my RSS reader’s search function to identify the articles that are of interest to me. I use a desktop reader call RSSOwl (http://www.rssowl.org/) but Inoreader (http://inoreader.com/) is a web based service that offers similar features and options.
Who is behind a site?
If I am to use any information from the web for business purposes I need to know who is behind the website. DomainTools http://www.domaintools.com/ is one of many services that will tell who owns a domain name, unless they are hiding behind an agent or privacy protection service. There is also a Whois+ extension for Chrome (my default browser) that can be used to run a quick and easy check on the domain name of a displayed page.
If you are interested in finding out more about business information resources I am running a workshop for TFPL in London on March 6th , June 6th 2014. Details are on the TFPL website.
Update: please note change of date for the next business information workshop. It is now being held on June 6th, 2014.
Delegates at yesterday’s business information workshop in London came up with an interesting combination of websites and search commands for their end of day top tips.
OFFSTATS – Official Statistics on the Web http://www.offstats.auckland.ac.nz Excellent starting point for official statistical sources by country, region subject or a combination of categories. All of the content in the database is in the public domain and available through the Internet.
GMackerhttp://gmacker.com/web/content/gDateRange/gdr.htm Google’s Verbatim in the search options menu on the results page is great. Google’s date option from the same menu is great. But you cannot use both together. You can use the daterange: command, though, with Verbatim but it’s complicated. GMacker is a much easier way to do it. Type in your search on the GMacker page, select your dates from the calendars and click on ‘Google Search’. When the results appear on Google simply apply Verbatim in the usual way.
Zanranhttp://zanran.com/This is a search tool for searching information contained in charts, graphs and tables of data and within formatted documents such as PDFs, Excel spreadsheets and images. Enter your search terms and optionally limit your search by date and/or format type.
News alerts, news curation services and automated newsletter generation. Use Google alerts, RSS feeds and newsletter generation sites such as Paper.li (http://paper.li/) and Scoop.it (http://www.scoop.it/) to keep up to date and share news with colleagues.
site: command Use the site: command to focus your search on particular types of site, for example site:ac.uk, or to search inside a large rambling site. You can also use -site: to exclude sites from your search.
Numeric range This command is unique to Google. Use it for anything to do with numbers – years, temperatures, weights, distances, prices etc. Simply type in your two numbers separated by two full stops as part of your search. A good way of limiting your search to pages or news covering a company’s activities over two or three years in the past.
OpenCorporates “The Open Database Of The Corporate World” http://opencorporates.com/ Provides access to open corporate data on 55 million companies in 75 jurisdictions. You can search all jurisdictions at the same time or select just one. Results can be filtered by type of data held, current status, company type, SIC. A link to the original registry page for a company is always included with the displayed information.
DataMarket http://datamarket.com/ A portal to thousands of free and priced datasets. Free to search, and create charts and visualisations of the free data.
I recently co-presented a webinar on researching legal information. The event was organised by TFPL, and Alan Blanchard and myself reviewed free and paid for resources together with key search techniques.
Throughout the session we polled the audience on a number of issues, the first question being “Does price guarantee quality when you are purchasing information?”. Surprisingly, given the topic of the webinar, 70% voted ‘No’ with the remaining 30% opting for ‘I don’t know’ rather than ‘Yes’. When we asked about their opinions on free information, though, 79% said they would need a result from a free source to be validated with a paid resource.
The audience could not qualify their answers – it was a simple yes/no/don’t know – but there were some interesting discussions on the issues after the event. The priced services certainly have to work hard to prove value for money and they cannot assume that their users will automatically renew each year. Free information has a big part to play in legal and business research but it is vital that one is aware of the limitations of free. For example, do you know how up to date legislation.gov.uk is and if it carries revised legislation? (See http://www.legislation.gov.uk/help#aboutRevDate for the answer). And then there is the issue of making Google run your search the way you want it run, without personalisation or deviation.
I am running two public access workshops this autumn for TFPL that look at free versus fee resources for business and legal information. The first, on 19th September 2013, is Business information: key web resources and covers:
Statista (http://www.statista.com/) is a portal for statistics, market data, market research and survey results. It aggregates data on over 600 international industries from more than 18,000 sources, including market researchers, trade organizations, scientific journals, and government databases. You can get a rough idea of what the service covers from the list of topics at http://www.statista.com/topics/, although this does give the impression that it is heavily US biased, which is not the case.
Searching Statista is straightforward. The default is to include what it calls word fragments, so a search on the term bank will also find banking. This can be changed to identical words if you require an exact match. To search for phrases place double quote marks around your terms, and documents containing a specific term can be excluded by prefixing the word with an exclamation mark. You can ‘OR’ terms by inserting the vertical bar symbol (|) between them, for example dog | cat. To refine your search further use the options on the right hand side of the results page. These include date of publication, region, free/fee-based/all, and category.
Trying to view the data and charts that you find can be a frustrating process as most of the information is only available on subscription. There is a limited amount of free information but you still have to register for what Statista calls its basic free access. The premium account for an individual user is USD 49/month and runs for a minimum of 12 months. This is automatically renewed so make sure you cancel before the year is up if you no longer wish to use it. The corporate subscription is USD 199.90/month. Again the subscription runs for 12 months. A free trial is available for the corporate service but not for individual users. Prices for universities, libraries and education are available on request.
On the results page you can easily identify the subscription content by the red plus sign next to the chart icon to the left of the result. A red ‘P’ indicates a priced report that is free to corporate clients but can be purchased by basic and premium account holders.
Clicking on a result takes you to a page where the data is displayed as a chart or graph depending on what Statista thinks is the most appropriate for that statistic, but you can change the representation if you wish. There are also options for downloading the data in various formats and links to the original source.
Information is provided on the source and includes title of the original document, the publisher, publication date, time period covered by the survey and the geographical regions covered. Statista also adds notes such as definitions of terms used in the original report and observations on possible missing data.
Much of the information on Statista can be found by via Google or Bing using advanced search commands and options, but you will probably have to wade through a significant amount of irrelevant information before you find the data you need. Statista can certainly save you a considerable amount of time, and the data displays are a great way of helping you identify the potential usefulness of a report, as can the metadata and notes.
Is the subscription worth it? It is definitely cost effective for corporates in terms of saving search time and ensuring accuracy and quality of information. I have had good feedback about Statista from several of my clients. I am not so sure about individual users. Some of my own research is highly specialised and I am not convinced that Statista covers all of the areas of interest to me. It is difficult for me to assess its value without trialing the full service first, and there are no pay as you go options such as a day or weekly rate. I might even consider paying for a month’s access for a project but it is a full year’s subscription or nothing. Nevertheless, I recommend you take a serious look at it and if you are contemplating a corporate subscription you can ask for a free trial. It is now on my list of essential research tools.
Towards the end of June I headed off to Swansea Central Library to facilitate a workshop on search tools and techniques for finding business information and statistics. The session was organised for the libraries of the wonderfully named SWAMP – South West and Mid Wales Partnership.
We had fantastic views from the library of the sea and shore line so they did very well to remain focused on the work in hand. The top tips that the group suggested at the end of the day were a mixture of search techniques and business information sites.
Don’t give up and don’t get stuck in a rut. If your first attempts fail to produce anything useful try a different approach to your search. Try some of the tips mentioned below: use advanced search commands, a different search tool or go direct to a website that covers your subject area or type of information.
Google automatically looks for variations on your search terms and sometimes drops terms from your search without telling or asking you. To beat Google into submission and make it run your search exactly as you have typed it in, click on ‘Search tools’ in the menu above your results, then click on the arrow next to ‘All results’ and from the drop down menu select Verbatim.
3. Private Browsing.
To stop search engines personalising your results according to your previous searches and browsing behaviour, find out where the private browsing option is in your browser (in Chrome it is called Incognito). This ignores all cookies and past search history and is as close as you can get to unfiltered results.
Short cuts to private browsing in the main browsers are:
Chrome – Ctrl+Shift+N
FireFox – Ctrl+Shift+P
Internet Explorer – Ctrl+Shift+P
Opera – Ctrl+Shift+N
Safari – click on Safari next to the Apple symbol in the menu bar, select Private Browsing and then click on OK.
4. The site: command.
Include the site: command in your search to focus your search on particular types of site, for example site:ac.uk, or to search inside a large rambling site. You can also use -site: to exclude sites from your search. For example, if you are searching for information on Wales and Australian websites mentioning New South Wales keep coming up include -site:au in your search.
5. The filetype: command.
Use the filetype: command to limit your research to PowerPoint for presentations, spreadsheets for data and statistics or PDF for research papers and industry/government reports. Note that in Google filetype:ppt and filetype:xls will notpick up the newer .pptx and xlsx formats so you will need to incorporate both into your strategy, for example filetype:ppt OR filetype:pptx, or run separate searches for each one. In Bing.com, though, filetype:pptx will pick up both .ppt and .pptx files.
6. Guardian Data Store (http://www.guardian.co.uk/data/)
For datasets and visualisations relating to stories in the news. This is proving to be a very popular site on both the public and in-house workshops. As well as the graphs and interactive maps the source of the data is always given and there are links to the original datasets that are used in the articles.
7.Company Check (http://www.companycheck.co.uk/)
Company Check repackages Companies House data and provides 5 years of figures and graphs for Cash at Bank, Net Worth, Total Liabilities and Total Current Liabilities free of charge. It also lists the directors of a company. Click on a director’s name and you can view other current and past directorships for that person.
10. Keep up to date Keep up to date with what the search engines are up to, changes to key resources and new sites. Identify blogs and commentators that are relevant to your research interests and subject areas and follow them using RSS or email alerts.
Companies and directors can be searched from the same search box. You then select the appropriate entry from a list of possible matches. The company summary is free of charge and includes a business overview, data on its status and business activities, and a list of trading addresses.
Also free of charge are the accounts and list of directors but you have to register (free of charge) to view the information. There are options for logging in with your Facebook, Twitter or Google account but if you prefer you can register a user name and password.
Five years of key financials (cash at bank, net worth, total current liabilities and total current assets) are shown as graphs and more detailed information is displayed in the Company Accounts Table.
The financial statements submitted to Companies House can be downloaded free of charge as PDFs. Other documents lodged at Companies House such as “Change of director’s details” or “Allotment of securities” are listed under the Documents tab and are £2 each.
The Credit Risk information (risk score, credit limit, payment data and key factors) and Charges (mortgages and County Court Judgments) are priced. For a single company the price is £4.99 + VAT, which gives you 30 days unlimited access to all premium credit data on that company for 30 days. If you are likely to be researching more than four companies on a regular basis it is worth upgrading to the All Companies options costing £20 + VAT. This gives you 30 days unlimited access to credit data across every company and director.
Current directors and secretaries for a company are listed free of charge. Previous directors and secretaries are part of the subscription service. The free director profile includes an overview, their registered details and a summary of the companies of which they are or have been a director. This can be more informative as a way of identifying connections between companies and other directors than looking at the company records in isolation. The full director report reports are £8.99 and include credit risk, CCJs, mortgages and charges, and a summary for each current appointment with key information taken from the associated company report.
For both companies and directors you can set up free alerts and add them to a dashboard. This is an easy way to compare results for companies, with negative and positive changes in key financials shown as red or green arrows pointing up or down.
Company Check is not the only service providing free access to some of UK Companies House and Ireland company data. DueDil (http://www.duedil.com/) and Bizzy (http://bizzy.co.uk) are two others that are worth looking at. I understand, though, that Company Check is working on additional services that are due for launch in the next few months. It will be interesting to see what they come up with.
When it comes to researching a market or industry sector the major national and international players are generally well covered by the established market research publishers. If you are looking at a highly specialised sector, are interested in smaller companies or just want to know who is doing what in a town or county then direct marketing lists can be the cheaper and better option. The source I often use is a UK based service called MarketingFile.com (http://www.marketingfile.com/). They have been around for a while but continue to update their services and ensure that the databases they offer are regularly checked and cleaned.
MarketingFile’s lists are divided into business and consumer marketing and further segmented by communication channel: postal, email, fax or telemarketing lists. You can drill down further and specify key job roles, industry sectors, geographic areas and company size. For consumer lists you can segment by criteria such as geographic areas, household finances, interests, investments, cars driven and charities supported.
For each list there is information on coverage, pricing and selection criteria.
You have to register to search the lists but registration is free of charge. The search screen makes it easy to select and combine multiple criteria and the number of results (counts), which is sometimes all one needs to know, is free.
The full data is charged on a per record basis and you can also opt to have only those records that include named contacts. If you are on a limited budget or want to test a sample of the data you can request a specific number of records for example 50, 100, 200, 1000.
MarketingFile have extended their services and now offer complementary services such as printing and posting of letters, postcards and inserts. They have a 100% Delivery Guaranteed offer on email and postal lists which offers 50p per item towards your postage for mail “goneaways” and 5p – 10p per item towards broadcast costs for email hard bounces.
It is not always easy to identify the most appropriate list for your research. Rather than waste time trying different lists I’d recommend that you contact their helpline, which is based in the UK. They will be able to give advice on the best strategy for your research or project.
Factiva is discontinuing its pay as you go option for occasional users as part of a “strategic refocus”. Instead of an annual fee of $69 plus a per document charge of $2.95, there will now be a fixed monthly fee of $249/month. This allows you to download up to 100 documents a month with additional articles above this limit being charged at $2.95 each. The new pricing will take effect on 30th April 2013 and, for existing customers, will be payable from their next renewal date.
I have not used Factiva for over a year and, to be honest, I haven’t missed it at all. Much of my research is highly specialised and I find that I am using industry specific sources and scientific papers more and more. Nevertheless, it is a great shame that Factiva has taken this step. Many of the people I work with appreciate the quality of the service but few use it often enough to warrant the $249 monthly fee.
Thanks to everyone who sent me details of the changes.
News and comments on search tools and electronic resources for research