Comment Articles

BSA figures do not add up

Tags:
Statistics,
BSA

Ingrid Marson ZDNet.co.uk

Published: 24 Jun 2005 15:15 BST

These are not the only misleading statements in the BSA study. Mingorance states in a later quote that: "European SMEs need their patents every bit as much as big companies."

I put it to the BSA that as only 2000 of around 11 million SMEs across Europe hold patents and as SMEs are responsible for half of Europe's turnover and employ more than 53 percent of Europe's workforce, according to the European Commission, doesn't the survey actually show that disproportionately few SMEs hold CII patents? I also asked how they justify saying that the need of SMEs is equal to that of large corporations?

The BSA responded: "This seems more than obvious. If a European SME takes the time and effort to get a patent despite all the well-known difficulties that they face, it's safe to assume they have good reasons and incentives for doing so."

Ambiguous and misleading statements is one thing, but surely the BSA would shy away from factual incorrectness. Well, it appears not.

The BSA admitted two weeks ago that their claim that "European small- and medium-sized businesses account for more than 20 percent of all CII patents granted since 1998", is incorrect, as the study does not distinguish between European and non-European SMEs. Despite realising this error the BSA decided not to sent out a corrected press release because "none of the stories that appeared that day…quoted the inaccuracy you had identified." Oh well, that's ok then -- no-one quoted the inaccuracy so it's ok to leave everyone sent the press release, including MEPs involved in making a decision on the directive, with false information.

The BSA failed to update the press release on their Web site until Wednesday evening, although they apologised for this. "For reasons we don't immediately understand, the press release, as you pointed out, was not corrected on the Internet – despite our explicit instructions that it should be," said the BSA's Mingorance.

The BSA admitted in a later email that in fact only 10 percent of European SMEs hold patents -- the others being primarily Japanese and US SMEs. "The implication of the correction we made is that 20 [percent] of the CIIs are held by SMEs and about half of these, or 10 percent, are European SMEs." So, quite a big mistake then.

So, maybe it was just the BSA's interpretation of the study. Maybe they're just not that hot at maths? Yet even the study methodology is flawed, and biased towards overcounting SMEs. The study took a list of what they considered software patents, got a list of non-SMEs and deleted every patent application from the list that had been filed by a non-SME. This method results in an "error of inclusion" i.e. non-SMEs will be left in the list, which the author admits. It appears much less likely that SMEs will be deleted from the list by mistake, but despite the obvious skew that the methodology has on the results, the author has not adjusted the data suitably.

There also seems to have been a considerable lack of due diligence on the author's part. He claims in his methodology that he also eliminated academic institutions and governmental organisations from the list, but in an hour spent examining the author's list of UK "SMEs" (the SME dataset was provided by the BSA), I found five universities: Aston University, University of Oxford, Royal Holloway University of London, Imperial College, and Royal Holloway and Bedford New College. A simple search on the word "university" and "college" would have highlighted those ones.

The author defined SMEs as those employing fewer than 250 persons and with an annual turnover not exceeding 50 million Euro, although in a footnote points out that due to the algorithm used the companies may have "considerably more than 250 employees". Again, a methodology that is flawed towards overcounting SMEs. In the hour spent looking through the list, I found eight companies that can certainly not be defined as SMEs:

1) ARM - the UK chip maker that in 2004 reported revenues of £152.9m
2) BTG International - a medical company that in 2004 reported revenues of £42.7m
3) DPT Global - a company that is part of the Carlyle Group, which describes itself on its Web site as one of the world’s largest private equity firms, with "more than $29.6bn" under management, a team of "nearly 300 investment professionals". The company also claims to have "committed more than $1bn of its own capital to its funds."
4) Macrovision Europe – this is part of the digital media company Macrovision, which had reported a net revenue of $182.1m in 2004
5) Infotech Software – the Indian outsourcing company, which employs 2000 people and had operating revenues of 1919.6m Rupees (36.4m Euros) Rupees from April to December 2004
6) NTL Group – the telecommunications giant that reported an operating revenue of £2074m over the last year
7) Revolution Entertainment Systems – this is part of the Leisure Link Group, which claims to be the "UK’s largest provider of public space gaming and entertainment machines" and employs "nearly 2,500 people"
8) Energy Pool Funds Administration -- this is part of National Grid, which owns and operates the electricity system in England and Wales. The company is owned by National Grid Transco, which is one of the UK's FTSE 100 companies, with a turnover of £8.5bn for the last financial year.

So, the author of the study somehow managed to miss a FTSE 100 company, a telecommunications giant and one of the biggest outsourcing companies in India.

There are probably many more big outfits there, but the dataset makes it arbitrarily hard to find out. For some reason it misses out the majority of company names -- only 193 out of the 606 UK listed patents indicate the company name. I looked up a few of the patents that had the company name left blank on the Esp@cenet Web site, assuming they were filed by an individual inventor, but actually found there was a company name listed. If you therefore take the simplistic approach of extrapolating the 13 non-SMEs found in a quick search through 193 company names, 6.7 percent of the "SMEs" are actually non-SMEs. That's one big error.

When I pointed out the five universities and eight non-SMEs found in the study, the BSA spokesperson responded. "Congratulations! With some investment of time and effort you have discovered what the BSA and the author of the study already knew and freely and clearly disclosed – that it is extraordinarily difficult for someone to screen out non-SMEs in a systematic way for the purposes of such a study."

Yes, but then this begs the question, if the study methodology is that inaccurate, why have you publicised the study? Or, at least, why didn't you adjust the data accordingly? Why have you used figures like SMEs account for "more than 20 percent" of patents in your press release, when in fact it would be more accurate to say "fewer than 20 percent" due to the skew in the data? Why haven't you mentioned the glaring problems with the study in the press release? How can a mere journalist find so much wrong in a couple of hours, when the people commissioning, preparing and publishing the study had so much more by way of resources and expertise?

It is difficult to conclude other than the BSA is happy to allow these errors because they fit the organisation's argument. It is also hard to conclude other than that the BSA studies really should be independently corroborated if they are to be trusted. As a lobbying organisation, it is doing itself a lot of damage by its attitudes and actions, and it risks damaging the very industry it claims to support by parlaying its inaccuracies into national policies and international law.