Editors, Jamie Bartlett and Carl Miller, introduce this technology edition of Demos Quarterly.

Publishing a collection themed on digital technology is difficult. Given the internet has now invaded almost every single aspect of our political, social, economic and personal lives, there are almost infinite things to write about. The lines between digital and not-digital are becoming harder to see.

We could have tried to crowbar the diverse and varied essays in this Demos Quarterly into some kind of manufactured theme. But it wouldn’t work. Digital technology refuses to be categorised. So this Demos Quarterly is as varied and far reaching as the technology itself. Infecting all and every walk of life.

If there must be a theme, it is this: this issue tries to translate and present the cutting edge research, boiling innovation and new thinking to the people that need to know what is happening, that need to make sense of events, and subsequently make, we hope, informed decisions.

But technology is also – to most of us – mysterious. Most of us have never seen an algorithm, yet our lives are in the grip of them every day. We sit with laptops on our knees and phones in our pockets, but what goes on inside those boxes might, to all but that tiny group of the indoctrinated, as well be magic. Look further out to the edges of emerging technologies, and it gets even worse. Immensely powerful technological storms are brewing, storms that could sweep away much of the furniture of our settled ways of going about things: our jobs, our institutions, our health, our politics – yet the effects they will have are largely unknowable even to the technology weather-makers themselves, let alone the rest of us.

In our view, technological advance over the last decade in particular has think tanks struggling to catch up. Think tanks, which used to be the home of forward and bold thinking, were – like many others – caught broadly flat-footed by the explosion of digital innovation, and are now struggling stay apace with the technology companies and digital start ups that are changing so many aspects of our lives.

That’s why we set up the Centre for the Analysis of Social Media at Demos: an effort to try to take some of the latest developments in pioneering ‘big data’ analytics and blend it with more traditional social science research methods. But beyond changing how research was done, we also wanted to start making sense of some of the dramatic changes taking place in politics and society as a result of digital technology. Technological advances can be bewildering, even quite terrifying. But, with some consideration, incredibly exciting – and bursting with potential.

This is what this technology edition of Demos Quarterly shows: a taster of the challenges, pitfalls and opportunities of the dramatic changes taking place across a wide array of areas. We don’t necessarily agree with all of what follows; but every single idea deserves your attention.

Ben Hammersley, in our lead essay, vents his frustrations and exasperations at the vacuum in leadership relating to all things digital, the inability or unwillingness of anyone political to live in the present, much less think about the future. Carl Miller is a little more optimistic, explaining that the transformation of politics is possible – in fact, it’s almost hard wired in to the basic design of the internet. Taking it all the way back to the English civil war, Carl paints an exciting prospect of new forms of democracy being made available thanks to digital technology. But he shares Ben’s frustration about where we are now. His illustration of MPs trying to ‘thunderclap’ on Twitter is as funny as it is depressing.

But someone on the front line has some good news. Catherine Riley from the Women’s Equality Party – almost certainly the most successful new political party in terms of interest and membership if not seats – described her party’s experience. Social media can and does provide a much needed communication tool for the upstarts. Especially if, as WEP did, they distribute responsibility for social media activity to members, rather than try to keep it centralised and hierarchical, as is the natural impulse of the larger parties. Opening up politics is certainly good for a healthy democracy. Newcomers are, given the scale of challenges we face, much needed.

If you were expecting to find a run of optimistic platitudes and tech-evangelising, of empty soundbites that the ‘here is now’, that ‘chaos is the new cool’, or that we are only limited by our imagination, you’ve come to the wrong place. Technology certainly hasn’t resolved all the world’s problems; and it’s started a few of its own. Those early 90s cyber optimists who thought digital communications would make everyone nicer have been impressively embarrassed. Holly Brockwell, founder and editor of Gadgette magazine, describes life a female journalist in the male dominated technology world and it doesn’t make for comfortable reading. In the wake of the Brussels attacks, Alex Krasodomski-Jones warns against the filter bubbles that surround us all on social media. There’s no such thing as a silent majority, just one you’re not tuned into.

Chris Montiero gives us the very latest insight into the latest fraudulent happenings on Tor Hidden Services (a.k.a. the dark net). For all the worries – and there are many – about internet censorship and internet bullying. Sir David Omand, former Director of GCHQ, examines what role the state has to play in dealing with some of these nefarious activities. It’s fair to say David Omand and Ben Hammersley probably don’t quite see eye to eye on this. But the debate on internet freedom and civil liberties currently occupies Parliaments in many democratic states. It’s become incredibly polarised, you’re either Orwellian or a terrorist sympathiser. But there are no easy answers here, and Sir David Omand,, sketches out what he considers a way forward. There’s something of a mea culpa here too, in that, in the past there was ‘a failure to explain to the public how the law was being applied, which is essential to comply with the spirit of the rule of law’. This year, he says, must be the year powerful surveillance capabilities are placed under the rule of law. Powerful spies; and powerful bodies to oversee them. Omand is of course the voice of someone close to the intelligence agencies: but that also means he understand better than most of us quite what they’re up against. His final word on that might give pause for thought (and concern).

What is both endlessly bewildering and exciting about digital (perhaps all) technology is the way it’s dual purpose. For every worrying application, someone’s works out uses that are positive and publicly minded. Sam Smith examines how the much hyped block chain technology – mostly associated with the cryptocurrency bitcoin – could be used to stamp out identity theft. Next week, we will publish a piece from singer-songwriter Imogen Heap, who paints an image of how the same technology could revolutionise the music, promising a fairer deal for artists in an increasingly sclerotic industry. Some of the most exciting – and fastest – development is technology is not here in Europe, but in Africa. Keren Elazari reminds us that hackers aren’t always the bad guys: in fact, they are essential for the future of a secure and safe internet. Nkundwe gives us a much needed whistlestop tour of how mobile technology in particular is transforming Africa, fast. Dr Amirudin Abdul Wahab from Malaysia also provides some grounds for optimism, looking at how social media is being used by various groups for social good, from areas as diverse as tackling bribery to providing clean water.

To imagine the far extremities of these competing visions – the optimist and the sceptic is the easiest way to pitch this – we’ve asked two radically opposing voices to battle it out. On the one hand, Zoltan Istvan, is a prominent US transhumanist, who (as we write) is running for President as a candidate for the futurist movement that hopes to use technology to end ageing, freeze our brains, and expand human capabilities. Against him is Richard Jones, head of Research for Sheffield University, author of a tract critical of the movement. Hardly a technophobe – a professor of physics – he worries too much tech-optimism results in misleading and even dangerous expectation about what science might achieve, both technically and morally.

Digital technology’s impact on our lives is often invisible, shaping what we do and what we see, without us fully understanding why or how. One of the most important roles a think tank (and journalism) might play in future is uncovering these new hidden powers and influences. Jeremy Reffin helps us understand one key new technology, and one like so many others now galloping forwards, that’s fundamental to the connection between computers and humans: ‘natural language processing’. Lurking in niche academia for decades, it’s now been thrust into the spotlight as the crucial way that big tech companies wring out the insight from the enormous bodies of data that they routinely collect. David Evans from the British Computing Society sets out how the opinions of those who make and run the technology we rely on really matters, but might sometimes be at odds with public opinion, with potentially important consequences. Jamie Bartlett concludes by linking these new technological advances to Demos’ core principle: power. Who has it, and how do they use it. While the internet has helped pull down some centres of power and redistribute the spoils to citizens, it’s erected new ones.

The task of the think-tank, and this edition of Demos Quarterly, is to help understand some of these new opportunities and challenges technology has created. It’s just a fraction of what’s going on out there. But running through all of it remains those central questions: how do we create a society together, balancing competing rights and interests in a way that permits commodious living. Technology doesn’t eliminate this concern, it simply re-frames it anew.