The news that HMRC is preparing to sell our anonymised tax data has made me wonder a little at how we will remember this decade when it is over.The reason I wonder is that, despite the launch of the Turing Institute - at least as a gleam in the Treasury's eye (and I'm sure my e-biography of Alan Turing was deeply influential here, especially now it reached #1 in one Amazon category last weekend!) - I'm not completely convinced that Big Data will really prove as important as we currently believe.
The reason is that it suffers from precisely the same weaknesses as any data. It gives the impression of hard numbers, evidence-based hardheadedness, objectivity and all that, but the numbers are also chained to definitions, and these are endlessly malleable.
The pursuit of Big Data is not going to be pointless, but it will be stuffed full of delusions - which won't be apparent until the decade of Big Data is well and truly over.
This thought has led me to have a think about the previous 'noughty' decade - the first decade of the twenty-first century - and the little delusions we used to tell ourselves. I've boiled these down in my head into three, and here they are:
Delusion #1. There is no need to make anything. The odd thing is that all our mainstream political parties bought into this delusion to some extent, that manufacturing was somehow below us in the UK - that we had somehow progressed beyond it. This was central tenet of the Treasury under Gordon Brown and Ed Balls, that financial services was the way we were going to specalise in the global marketplace - without realising that financial services was also the cuckoo in the nest: it eventually drives out everything else. Thanks to Vince Cable and others, UK manufacturing is slowly being clawed back. So that little delusion is, I think, over - even if we still haven't understood some of the practical implications for the kind of financial services we need. The current financial sector is wholly ineffective when it comes to providing for the capital needs of manufacturing.
Delusion #2. There is no such thing as geography. This was a delusion foisted on us by the internet and AI pioneers of the US West Coast. You couldn't be in any kind of policy meeting without someone saying something fatuous about 'online communities' being the future of community. A decade later, I think most of us realise that, if we believe our Facebook friend are our real friends, we are likely to face a lonely future. The truth is that geographical communities, awkward and authentic as they are, are the only effective ones, for most of us - though of course an occasional exchange online can't be a bad thing.
Delusion #3. Everything can be measured. This was the greatest delusion of them all - the quotation is taken from the ubiquitous management consultancy McKinsey - when the truth is that the really important things, from love to happiness, are wholly unmeasurable. This delusion still has many of us in its grip, and especially as we hurtle into the decade of Big Data, which suffers from the same kind of drawbacks.
Of course, in practice, these were also the delusions that were written into the New Labour years like the writing through a stick of rock, and this is where it matters that we are still in the grip of delusion #3. In practice, we subjected our services and voluntary sector funding to incisive questioning over their impact, asked them for evidence, and then handed the money over to institutions that were able to provide it.
It would be quite reasonable, were it not for two problems. A huge percentage of the time, energy, creativity and effort was shifted from making things happen to producing data, which was flawed. Of course it was flawed: people's jobs and livelihoods depended on it being good.
The second problem was that we handed services increasingly over the new outsourcing giants, because they were able to produce reliable data about their impact, and then sat back and breathed a sigh of relief.
It is slowly becoming clear, unfortunately, that the production of data was the only thing they could do. It was the sole purpose of those great outsourcing monsters, the central purpose of their design. It seems likely that they can produce the data, but can't actually have an impact - and this very distinction was what was being denied.
So we remain in the grip of at least one delusion of the previous decade. Here's to its speedy unmasking...