Obligation. Responsibility. Culpability

A conversation with a friend a few days ago has made me think about another frame in which to consider action and effect. One which acknowledges the complex, entangled nature of our social relationships and the delicate balance of forces that hold that web of relationships together.

To whom do I have (mutual or reciprocal) obligations and what are the nature of those obligations? Will my actions demonstrate my enacting those obligations or an avoidance of them?

What are my responsibilities to those with whom I have relationships? How do I balance my needs and any impact my actions may have on them?

Am I willing to accept the consequences of my actions? Am I willing to amend my behaviour in future to reflect any detrimental impact of my actions on others?

In these muddled times, full of fear and anxieties, being able to think clearly, to make informed choices and decisions seems ever more urgent and necessary. Understanding that we are all connected in multi-lateral and multi-dimensional ways, and to have the critical means to make assessments that acknowledge this instinctively, is like a ray of sunlight cutting through the gloom.

Weighing opportunity and intent

As our world plunges into an era of uncertainty and reactive measures against plurality, diversity and tolerance I have been trying to formulate a framework or set of conceptual scales to guide my choices in life and work. How can one weigh out the opportunities that appear? How can intention and action be balanced in a way that respects our essential ethos, whilst acknowledging the constraints and compromises that are necessary to survive in our challenging social, political, economic and cultural order?

What I have realised over the years is that my work has always been about the kinds of human relationships that are forged within it. All my work is collaborative – often crossing disciplines, skills, trades and sectors. What I seek is a quality of experience generated by the energy of a group of people engaged on a collaborative or cooperative enterprise. The outputs (events, tools, artworks, publications, films, media, installations etc) are indeed important – primarily to share the experience with others – but are not the main driver for the work. It is people and the relationships which I develop with them that are my true inspiration and motivation.

In seeking to make judgements about what projects to contribute my energies to, what sorts of opportunities to explore and which kinds of trajectories to pursue, I have begun to identify the essential qualities I want to be present. These are not universal or rigid measures, but rather an attempt to have a simple rule of thumb to gauge whether or not they balance in any given configuration: Passion. Intensity. Intimacy. Pleasure.

It is important to me to work with people who are passionate and care deeply about what they are doing, why they are doing it and how they go about it. It is the lifeblood of a team’s commitment to a project, practice or idea and without it it there is often no bigger vision or aspiration.

I find that the most satisfying work is produced at a high level of intensity. Not constant, but following cycles and rhythms within the natural flow of a project’s development and the broader patterns of life that surround us. The ebb and flow of such work cycles naturally generates as well as consumes energy, much as we find in physical exercise.

I judge the richness of my life not in the material things which surround me but in the nourishment I get from close acquaintance and shared bonds with other people. These connections are vital to feeling that I am part of the world and of various communities. It is through other people that I discover new things, new joys and pleasures; and share the things I value and cherish with others. Working together to define a project, identifying problems or issues to be addressed creates extraordinary opportunities for people to open up and create common feeling across social and cultural divides. Without some degree of intimacy with one’s colleagues we are often mired in isolation and loneliness, even in the midst of others.

What kinds of pleasure will flow from this endeavour: the pleasures of association with old or new colleagues; of new skills acquired; of ideas challenged or evolved; of successful reception of outcomes and outputs; of new directions indicated or taken; of abstract thoughts made manifest and shared? Without some sense of pleasure in and from work it soon degenerates into a self-perpetuating drudgery of obligation to a status quo. For the sake of human spirit I think this is neither sustainable nor resilient in the long term.

As individuals in complex societies, our abilities to effect positive change that counter destructive forces often appear insignificant in the face of such huge systems and overwhelming structures. But exercising informed judgements about what we do and why is surely one of the best and most profound ways we can retain a sense of personal agency against the tide of narrow automated, rule-based decisions increasingly foisted upon us from above. Nurturing a plurality of informed judgements made by ordinary people in their daily lives might offer a powerful corrective to the strictures and conformities now being insisted upon by autocracies of all kinds and natures.

Exploring Risk


Over the past 6 weeks I’ve been working with Professor Lizzie Coles-Kemp and her team in the Information Security Group at Royal Holloway University of London to produce a publication as a deliverable for their part of the TREsPASS project.

TREsPASS : Technology-supported Risk Estimation by Predictive Assessment of Socio-technical Security was a 4 year European Commission funded project spanning many countries and partners. Lizzie’s team were engaged in developing a “creative security engagement” process, using paper prototyping and tools such as Lego to articulate a user-centred approach to understanding risk scenarios from multiple perspectives. The three books and the poster which comprise TREsPASS: Exploring Risk, describe this process in context with the visualisation techniques developed by other partners, as well as a visual record of the presentations given by colleagues and partners at a Summer School held at Royal Holloway during summer 2016.

The publication has been produced in an edition of 400, but all 3 books included in the package are also available to read online via bookleteer, or to download, print out and hand-make:

We are now starting a follow on project to develop a creative security engagement toolkit – with case studies, practical activities and templates – which will be released in early 2017.

Fairness and Bias in an Algorithmic Age


Last month a new research project of which I am part got underway – UnBias: Emancipating Users Against Algorithmic Biases for a Trusted Digital Economy. Its a collaboration between the Universities of Nottingham (Horizon Digital Economy Institute), Edinburgh (Informatics) and Oxford (Human Centred Computing) funded by the EPSRC through its Trust, Identity, Privacy and Security in the Digital Economy strand. Over the next two years it will look at the complex relationships between people and systems increasingly driven by personalisation algorithms and explore whether, and to what degree, citizens can judge their trustworthiness.

My role will be to lead a co-design process that will create a ‘fairness toolkit’ : raising awareness about the impact of algorithms on everyday behaviours; devising pragmatic strategies to adapt around them; and engaging policymakers and online providers. We will be working with schools and young people to co-develop the toolkit – following in the wake of previous projects exploring young people and social media, such as Digital Wildfire.

For me this project cuts to the quick of concerns at the heart of today’s society: empathy, agency, transparency and control. I will be bringing ideas and practices to the project I have been exploring from a number of different trajectories over the past few years, from my work on the Pallion project to data manifestation and reciprocal entanglements. I am particularly excited as this marks my first formal collaboration with Oxford’s Human Centred Computing research group with whom I’ve been in dialogue for a couple of years.

Mapping Rehab Journeys


Back in the summer of 2014 I devised a simple paper-based tool to help brain injury survivors map their rehabilitation journeys through a gentle creative process. It was part of the work I did for the Movement Science Research group at Oxford Brooke’s University – helping them to understand how to create a patient-centric digital rehabilitation monitoring tool. Having tested it successfully with a group of brain injury survivors in a workshop at Headway Oxford, it was frustrating not to be able to find a way to develop my ideas into a simple, low cost and flexible tool that could be used by brain injury survivors to periodically record their experiences of rehabilitation.

Since then I have been mostly engaged on a project co-designing a simple method and tools (based around bookleteer.com) for documenting Traditional Knowledge with villagers living in the jungle in Papua New Guinea, as well as with my work on data manifestation. As the notebooks for documenting knowledge have evolved, I have seen how they enable people to capture things that they value in ways that help them communicate and share that value further. Reflecting on this over the past few months, it made sense to think about adapting the ideas for the rehab journey tool into a booklet – utilising the simplicity, low cost and ease of use of the bookleteer format, something requiring no immediate investment beyond my time.

I was also inspired to do this having seen the excellent Patient booklet co-created by Grace Tillyard as part of her Breast Cancer Awareness project in Haiti. I have been providing a small bit of advice to Grace over the past year and half as she developed her programme, and we had talked about using the bookleteer booklet format as a simple way for people to record their experience of cancer treatment and recovery, as well as providing them with information about the process and possible outcomes. Her use of bookleteer to create a powerful tool that can be used effectively in the complex socio-economic environment of a developing country is truly inspiring.

Having discussed my idea with my friend and colleague David Sinden, who co-facilitated one of the workshops in Oxford with me for the TBI project (and who has himself survived a brain injury), I have now created a draft notebook format to help people record their experiences of the rehabilitation journey. It develops on from the original worksheets, encouraging survivors to use both words and drawing to document how they feel about their journey of rehabilitation in the present moment. The prompts ask for feelings related to time, progress, fluctuation and speed of rehab, as well as an overall mapping of the journey itself.

As with the notebooks for documenting Traditional Knowledge, I felt that it was important for people to be able to personalise their notebooks by putting a photo on the front cover. This might also serve as a visual reminder of how someone was feeling at the time, as well as how they look (or, for instance, include the face of someone who might be helping them fill out the book, such as a carer or friend). This can be easily done using the same sort of low cost kit we use when in the jungle of PNG – a basic cameraphone and Zink photo printer (e.g. the Polaroid Zip printer or Snap camera/printer).

What might the benefit be?
When I ran the rehab journey mapping workshop with five brain injury survivors at Oxford Headway in August 2014, they all became very invested in the process and were vocal in expressing their enjoyment of the task. It was highly social, with a lot of mutual support (especially in remembering things for each other) and free-flowing conversation which in turn provoked considerable self-reflection. I acted as the scribe for one of the survivors, assisting with writing and drawing for her. At the end of the workshop she thanked me and said that this exercise had been the first time anyone had sat down with her and listened to her describe her experiences since she had suffered a stroke about 12 years before. I asked the group if they thought it would be worth doing the exercise again, perhaps on a fairly regular basis (such as two or three time a year). They all agreed that it would be worthwhile, not least because it would give them a sense of how their feelings about their rehab journey change over time and this would give them a way to recall how they have felt at different times.

It is this aspect of self-observation that seems to hold a great deal of value in the process – not just for personal reflection, but also when discussing an ongoing rehab journey with the clinicians, physicians and other professionals and carers who are supporting survivors. Building up a collection of these observations over time could help reveal patterns that often evade us as we live in the moment. They are a form of data that could be useful for longitudinal analyses of the changes and adaptations experienced by people over prolonged periods of rehabilitation.  They offer the potential to collect and collate rich, qualitative information about how rehab is experienced by the survivor themself which could enhance other quantitative data already being collected as part of ongoing care and health management.

The draft notebook is available for anyone to try out in two sizes: Small (A6) and Large (A5). This is just a first step and is in need of feedback to both be improved, and built upon – as there are undoubtedly other ways and tools which could add to this. David and I will be collaborating on testing and extending the format as well as devising a practical process or methodology for both health professionals and peer groups of people experiencing rehabilitation to use it. We hope to have more news about this in the Autumn.

For some new Putney Debates

Earlier this year I was shortlisted for a residency in Lincoln as part of a project celebrating the 800th anniversary of the Magna Carta. My proposal (“Lincoln Debates“) was to work with young people in the city to re-evaluate the state of our society and to bring their voices concretely in to discussions of where we go next. I proposed to engage people in gathering and sharing their voices – through collaborative public authoring and the making of material artefacts, much as I do in my work with indigenous communities in Papua New Guinea and my work with data manifestation. Embedding their concerns and aspirations for civic empowerment within the fabric of the city through events and with physical, digital and other media I hoped to devise a project that could resonate for future generations and inspire in them a continual engagement with power, rights, obligations and responsibilities.

1647 Putney Debates
My project title alluded specifically to the debates held in October 1647 at St Mary’s Church, Putney between the representatives (“Agitators/New Agents”) of the rank and file of the New Model Army, and the Army Grandees (Oliver Cromwell, Henry Ireton etc). The debates were part of a proposed settlement towards the end of the English Civil Wars to determine a new future for England (and by implication Scotland, Wales and Ireland). The men of the army had developed their own manifesto, which would later become a manifesto for a proto-socially democratic constitution for England – An Agreement of the Free People of England. Their demands were essentially communitarian and advocated for much of what we would now recognise as the basis of a functioning democracy, but where there would be no inherited privilege (thus ‘levelling’ each person’s status). The Debates were ultimately unsuccessful, but the discussions that took place and the manifestos that resulted became the inspiration for future milestones in our own and other country’s path to freedom and democracy.

Modern Putney Debates
As we lurch forwards into a new time of uncertainty and division in this country, it seems to me that we could do with more such debates and proposals – not merely as an exercise in creative agency, but vitally as a positive beacon for the future of our country, or countries, our identity and our aspirations to do good in the world.

What would it take to organise some events that draw inspiration from the Levellers of 1647 and strive to propose a new social contract founded on equity, compassion, responsibility, empathy and agency?

Last year I published a series of books that brought together key texts that derive from Magna Carta and the Charter of the Forest – spanning 900 years they represent a stunning legacy of hope against oppression, subjugation and exploitation. Now is the time to start adding to that legacy with something new.


Data Manifestation Talk at ODI

Video of my talk at the Open Data Institute on Friday 17th June 2016:

For further details, read my post, How Do We Know?

How Do We Know?

The following is the text of a ‘provocation’ I gave at the Responsible, Ethics-Aware Research and Innovation in Data Science symposium hosted by the Alan Turing Institute in March 2016. For the event, my colleague Stefan Kueppers and Professor George Roussos at Birkbeck University of London generated a new set of Lifecharm shells using data from a research trial with sufferers of Parkinsons Disease. We 3D printed multiple copies of 4 shells which were given to the participants as a tangible souvenir of the data manifestation concept.


I’m going to speak today about a concept I call “data manifestation” and how I believe it can add a significant dimension to data science and how we make meaning from data.

First I wish to pose a question –

How do we know what we know?
And how do we make meaning from what we know?

The answer is, of course, through the interaction of our senses with the stimuli coming in to them from the external world – and how this is processed by our consciousness and woven in together with our memories and emotions.
This is a system of high complexity and entanglement. It produces judgements based on multiple factors and dimensions all working together.

I believe that this is qualitatively different to the kinds of inference driven systems that are increasingly coming to dominate our civilisation and how it operates.

For instance: when we feel that something we have eaten is delicious, how do we know this?
It is not just the taste that defines this judgement, but includes many other factors such as smell, look, texture, sound, shape and size, whether it invokes memories, or causes a momentary shift in our sense of balance. If each of these factors is seen as a data input, how would we begin to map such complexity into the decision-making systems of today?

Art and aesthetics are another example of this complexity. When you encounter a work of art the experience itself which determines your aesthetic reaction to the piece – whether one of awe, delight, revulsion or indifference – is driven by similar complex factors. The use of materials, colour, scale, lighting, etc as well as memory and emotion create each person’s own aesthetic experience. There can be no right or wrong aesthetic experience, only how each individual experiences a work of art in relation to the summation of their own existence.

Much of the data that is captured, recorded, analysed and used as the basis of inferential decision making systems in the everyday world is principally derived from visual interfaces, from the writing of the software that generates it to the representation of the data in screen-based visualisations – from spreadsheets, to graphs and animations. There are in a few cases some sonic representations of data, and in extremely rare cases some haptic interfaces. But on the whole the way we create, analyse and present data is visual and screen-based.

But not all experiences are visual. And my proposition is that by failing to encompass the whole and extraordinary sensitivity and careful tuning of the human sensorium in our methods of expressing data we are missing huge potential.

Back in 2012 Proboscis was commissioned to collaborate with scientists at Philips Research Lab in Cambridge as part of a public art programme. We were asked to explore the problem posed by the failure of commercial biosensor devices to engage people in leading more healthy lifestyles. Essentially the FitBits, Fuelbands and similar devices were being switched off within a few weeks of activation. Philips is a big player in the separate business of TeleHealth (where people with serious medical conditions have instrumented homes that record and transmit data to health professionals and back to the users). They were interested in how these emerging life-tracker devices could have positive well-being and health benefits for nominally healthy people over the long term. But the usage data suggested this was simply not happening.

Looking at the problem, our insight as artists was that this was an issue of relevance to people’s sense of their own self, to their identity. Our hunch was that the ways in which the narrative of a person’s life helps construct their sense of self were just not being engaged by the graphs of data represented on smartphone screens or laptops. Humans frequently invest meaning in objects that act as triggers for memory and personal significance. These can be highly ritualised objects such as wedding rings or other kinds of jewellery. They can also be seemingly random objects, such as a piece of cloth or a pebble from a beach. With all these things, the memory is evoked not just by the sight of the object, but through other factors such as the way it feels.

Our insight was to connect this tactile relationship to meaning with the kinds of data that could be collected by tracking devices. We built a simple data logger with off the shelf components and collected a range of data over a week from members of our studio: step count; pulse rate, sleep patterns, blood pressure, stress factors etc.

My colleague Stefan Kueppers then developed an algorithm based process to flow this data into a shape that could be 3D printed. We looked to nature for inspiration and realised that the mathematics and geometries in shells would allow for extraordinarily complex and unique expressions of data in a 3 dimensional form. Here are a few examples of the objects we made from our own biodata. We called these “Lifecharms” and the project, “Lifestreams”.

What I am going to show you next is our latest experiment. We have been collaborating with Professor George Roussos at Birkbeck (where Stefan is currently working on his PhD about the Lifestreams work) to adapt and extend the data manifestation concept to other uses.

Birkbeck have been running a data collection project working with Parkinson’s Disease patients – mapping the wide variation of symptoms that they experience. Parkinsons is a disease encompassing high variability in symptoms between individuals; ideally this ought to imply a high degree of personalisation in their medical care and disease management regimes. Yet health policy and medical care management for Parkinson’s sufferers are determined by a ‘scale’ (called the UPDRS) derived from about 70 factors of motor performance criteria a Parkinsons patient is classified against. These factors are collapsed into a single “summary statistic”, which is then used to assign treatment for patients according to where they lie on the scale. Whilst this is an effective way to communicate the multiple dimensions of Parkinson’s on a linear scale of progression, because of the enormous variation it is common for people with entirely different symptoms to score similarly on the scale, yet require very different care.

For patients and health professionals alike, finding ways to express the unique characteristics of an individual’s actual experience of Parkinson’s would be a big step both for personal dignity and as a way to convey the wide variations in an easily appreciable manner to policymakers.

What you see in these four examples here, a version of which you are being given to take away and keep, is an expression of three data sources from four individuals who participated in the Parkinson’s trial, each mapped in 3 dimensions into the form of a shell. They demonstrate the clear variations in symptoms experienced by these individuals. The data used were :

1) Tremor in the Left Hand – which maps to how much the spiral of the shell stretches outwards.

2) Left Leg Agility – which maps to the overall growth scale of the shell (i.e. how big it gets in terms of volume)

3) Two Target Fingertapping (Left Hand) – which maps to the change in the frequency of the recurring ridges on the exterior of the shell.

These four shells are only a very initial experiment and there are many more factors that we will be able to control with data sources from the trial in future iterations, such as the number and increment of rotations, ripples in the curve of the shell exterior, the sweep curve of the main shell, twists in the shell curve, other geometric entities occuring on the shell surface (spines and nodules) as well as the number of segments that make up the whole shell.

A key indicator in Parkinsons is how it affects a person’s left and right sides of their body so, in the future, we also propose to develop twin or “clamshell” forms that spiral out from a common centre and reflect differences recorded between a person’s left and right symmetries.

As you can see, and soon feel, even with just 3 data sources informing the shell growth parameters, it is possible to appreciate the individuality of each person in a way that is qualitatively different to looking at a graph. As we introduce more granularity and complexity, the individuation will become even more pronounced.

Taken as snapshots, such shells, could over time reflect the changes in Parkinson’s symptoms as experienced by sufferers.

What use is this? We believe that this kind of approach offers a very different and rich way to express complex datasets for making meaning in a world where data is ubiquitous, is increasing exponentially and – as we are constantly being told – is overwhelming us.

My proposition is that in this way we can find alternative ways to make meaning and take decisions based on human insight and judgement from complex data sets. We don’t just have to simplify and summarise data in linear ways to make it easier to represent visually on a screen, we can also use our other senses – touch, sense of scale and balance, taste, smell, hearing, time and temperature. We can also benefit from other knowledge traditions (as well as contemporary science) and apply methodologies and critical analysis from the arts and humanities, such as aesthetics, to make meaning and draw judgements from highly detailed material artefacts that reify complex data sets into actual things, not just representations on a screen.

Thank you.

Characteristics of the Shells

Shell 1 – spiral not so pronounced, which indicates that the left hand tremor amplitude is not so strong; the shell size is quite big indicating the patient has significant increase in amplitude of left leg tremor; the ridges are also become quite pronounced indicating that there is a growng lack of accuracy and slowness in finger tapping.

Shell 2 – the spiral is at the higher end, suggesting the patient has a higher amplitude of left hand tremor; the shell size is a bit smaller so the patient experiences less left leg tremor amplitude; the ridges are very small indicating a higher degree of accuracy and speed in the patient’s finger tapping.

Shell 3 – the spiral is a median, indicating a moderate but not pronounced amplitude of left hand tremor; the shell size is slighty above average, suggesting increasing tremor in the left leg; and the ridges are fairly pronounced suggesting a greater lack of accuracy and slowness in finger tapping.

Shell 4 – very pronounced spiral suggesting very high amplitude of left hand tremor; shell size is smaller indicating less amplitude of left leg tremor; the ridges are very pronounced suggesting a high lack of accuracy and slowness in finger tapping.

Reciprocally Entangled

This statement of my current thinking on the topics of agency, data and quantum computing was originally presented to the Human Centred Computing research group at the University of Oxford on October 1st 2015. The group, with whom I have been developing a dialogue since Spring 2014, is led by Professor Marina Jirotka and has a particular focus on Responsible Research and Innovation. They are involved in embedding RRI into NQIT – the Networked Quantum Information Technologies Hub based at Oxford. I have made a few changes to the original text inspired by the group’s extremely thoughtful comments and discussion.

At the core of all my work and projects over the past 20 years is the theme of agency. By that I mean our ability as humans to act on our own initiative, to make informed decisions and choices. To be willing actors rather than directed subjects in all areas of life. In my experience open and transparent communications are fundamental to such an aspiration, and many of my projects explore ways and means of enabling people to communicate and interact with other people and systems to achieve this.

Looking back, there is a clear trajectory in my work which emphasises that systems should engage people and empower them, not seek to condition their experiences or shape their lives. From my work on the convergence of mobile phones with wireless internet and GIS mapping technologies in the early 2000s which has been described as a kind of proto-social media (Urban Tapestries), to experiments with sensors and platforms for citizen science style pollution mapping (Feral Robots & Snout), as well as experiments in data manifestation as a critique of the quantified self meme (Lifestreams). In my opinion, too often the technologies that are deployed across society are not intended to benefit all, but are contingent on bringing benefit to a privileged few – those who build them and those who commission and own the systems being built.

I believe that human individualities and different people’s idiosyncracies of learning and understanding are crucial social assets that enrich our collective experiences of life. I want to explore how we can design systems that adapt to these values and incorporate their dynamic into their very fabric. I want to find alternatives to the expanding deployment of systems and technologies that shape and manage behaviour by imposing rigid and inflexible decisions or choices. I believe that we need to question and challenge the mindset that sees the efficiency of algorithms and data-driven inference as the pinnacle of how human societies should be run.

One of the ways I think we can begin to revolutionise our relationship to data is by bridging the biological and digital. If we can design ways to utilise the whole human sensorium (not just vision and hearing) for sense-making and interpretation, then I believe that we can make complex information tangible and appreciable in richer and more nuanced ways. This means departing fundamentally from normative data representations on computer screens. It means embodying them in reciprocally interactive engagements that afford us greater use of our highly developed senses – what I have been calling “data manifestation”. This could allow us to experience data in ways that reveal things we have hitherto not considered possible. It may also reveal contingencies and limitations in what kinds of data are being collected – and may lead us to collect different kinds of data that have perhaps been overlooked.

How might we enliven relations between humans and machines so that they can be mutually influential rather than unbalanced in favour of one side or the other? I suspect that the promise of quantum computing, with its multiple states, may offer something along these lines, but only if we, as researchers and designers, have the courage of our imagination to make such a future possible.

Entangled Engagement & Quantum Computing
I have been thinking recently about the nature of entanglement – as far as I can understand it as a lay person not a scientist – and how the highly metaphoric language itself suggests an unintentional or, at least, ambiguous state. If we think about how we use the word ‘entangled’ in everyday situations it is to describe something, or some things, that have become entangled accidentally – without the deliberate decision to do so.

I have titled this statement “reciprocally entangled” because I think that the promise of quantum computing is one we must have agency with and choose to engage with, rather than unwittingly becoming caught within. That we might deliberately choose to enmesh ourselves with a system that, like humans, can be more than a cascade of rules and simple on/off decisions, would be a significant revolution in how we decide to run our societies.

I think that this idea of ‘unwitting entanglement’ characterises many of the fundamental problems we are experiencing with the so-called Big Data revolution, where we have suddenly begun to find ourselves subject to systems that are ruled by inference and not by balanced judgement. My proposition is that just as we might think that our society and civilisation has become enmeshed in a complex set of interlocking inference systems which define people’s lives in ever more intrusive ways, so might we begin to think of quantum computing – and being reciprocally entangled with such a system – as being closer to the kind of complex and consciousness-driven judgement that previously defined our choices and decisions (since, say, the Enlightenment).

More and more, the effects of decisions made by data-driven inference are coming to pervade aspects of our everyday lives. It is not always clear how such systemic decisions are arrived at, but it is undeniable that people are both shaping and having their choices shaped as an affect of the increasing reliance on so-called Big Data systems. A kind of blind self-governance that could so easily tip over into forms of self-censorship, self-privation and self-denial.

Such choices are most visible to us on an everyday level in the recommendations we see in internet shopping (“you’ve bought this and might like that”) and what social media systems select to populate our profile feeds (both adverts and posts by our connections). Their use in electronic financial services is widely known of, but little known about – obscured behind a veil of exclusivity, secrecy and the disparities of wealth and power. The role of such systems in health is also increasing, driven as much by attitudes towards risk and liability in the health insurance industry as by advances in medicine, wellbeing and disease prevention. These opaque applications remain worrying precisely because they flow against the transparencies of fairness and democracy that our society has been implementing over the past few centuries.

But what if we can rethink how we interact with systems as reciprocal engagements? And what if we were to see them as entangled relationships at the same time?

Perception is more than simply seeing
How humans perceive and create meaning is an associative process that is fundamentally different to inference from data. It is an expression of the difference between consciousness and a structured system; and, furthermore, it is more than what is offered by systems that mimic ‘neural nets’. Aesthetics and how we make meaning from artworks gives us a concrete example of how this operates in practice. Aesthetics cannot be taught, but flows from a dynamic interweaving or perhaps an entanglement of our memories and experiences with our physical senses. At each moment of experiencing something in an encounter with an artwork – writing, painting, sculpture, music or performance – we are existing in an entangled moment blending the now with the summation of our conscious sense of self. We are not inferring meaning (as a structured data-driven system might), but actually making meaning from the experience itself.

So-called neuro-aesthetics has sought to find a physical explanation of this process in the chemical reactions in the brain. But so far it has not been able to,

“ Objects are not triggers for internal events in the nervous system; they are opportunities or affordances for our continuing transactions with them. …
Art is experienced in the setting of argument, criticism, and persuasion. This is all compatible, Kant realized, with the fact that there is no way of adjudicating disputes in this area, that there are no decision procedures, no rules, no way of proving who’s right and wrong.”
Alva Noë, How Art Reveals the Limits of Neuroscience, The Chronicle Review

Beyond Visualisation : Embodied Entanglements
What if we can bridge the digital and biological by utilising the whole human sensorium for sense-making and interpretation – going beyond just vision and hearing? What if we could make complex data tangible and appreciable to a range of human senses? Embodied and felt instead of just seen and heard. In turn the experiences could be fed back in to data systems as a new set of parameters that could adjust the nature of systems that use the data. This was the theme of my Creativeworks residency with the Computer Science department at Birkbeck in Autumn 2014. The workshop I devised to help communicate the potential of multisensory expressions of data has been a crucial step in continuing to develop the ideas begun with the Lifestreams collaboration with Philips Research in 2012.

What I hope to do next is to embark on a journey of collaboration and discovery to demonstrate the potential for social change that could be unleashed by developing multisensory interactions with the digital data that is increasingly measuring, being analysed and governing our daily lives. Humans have extraordinary sensory capabilities which are not currently being used in how we encounter data – principally through screens (sight), occasionally sound and, rarely, through haptics (touch). I believe that this leaves us impoverished. By expressing digital data in new forms we could unlock entirely new modalities for recording, sharing and understanding how we live our lives : from experiences of illness and rehabilitation via biosensors to how we make sense of the Big Data that now shape and govern our society.

I hope to explore this theme along a series of trajectories and to demonstrate – to scientists, technologists, designers and policymakers and to the wider public beyond – how we could create transformational ways for people to engage with and make sense of data. I aim to work collaboratively (with old and new partners) to develop projects and experiments that express digital data in different ways that engage human biological senses other than just sight and hearing – such as touch, smell, taste, balance, temperature, proprioception and time etc. I am particularly keen to engage with researchers in the biological and life sciences (and hybrid fields such as computational biology) to create a bridge between how people interact with and make sense of the biological world around us, and with digital systems.

Documenting Traditional Knowledge in Papua New Guinea
Alongside these concerns I am working with Professor James Leach and a community of indigenous people (from Reite village) living a traditional way of life in the jungle in Madang Province, Papua New Guinea. There we are co-developing a simple hybrid digital/physical toolkit for people to self-document local traditional knowledge of plants, customs, design and productions techniques (TK Reite Notebooks). The tools and techniques we are co-designing there are based on my pioneering work since the late 1990s in hybrid digital/physical publishing. It uses the Diffusion eBook format, a unique paper folding method invented in 1999-2000, as well as the bookleteer.com self-publishing platform which was created and launched in 2009. This is another form of what I have called “public authoring” – making it possible for people to communicate things of value in open and shareable ways.

What is clear is that Reite people’s culture doesn’t consider knowledge as a series of static objects – in the Cartesian tradition of Western Knowledge – but as relational. Knowledge is constructed and exists in the relations between humans that hold it, share it and pass it on to future generations. It is always contextual and situational. To me, this establishes a conceptual link with what I have been exploring in my data manifestation work – that how we develop understandings of information and the ways it can be encoded in forms or media is located in the experience of the encounter, and our conscious ability to reweave our experiences in the present with those in our memories of the past.

One of the most striking things about this project is the difference between traditional anthropological ethnography and self-documentation that emerges from its results. What people choose to value and document may not necessarily be intelligible to outsiders, or of any interest. This flows against the mainstream of anthropology and ethnography which seeks to observe, understand and explain a culture to others. I feel that my conception of public authoring is akin to the aims of the originators of Mass Observation, who proposed an “anthropology of ourselves” back in the late 1930s. They thought that it was important for ordinary people to document ordinary life, and developed a framework that encouraged people from all walks of life to record and submit reports on things, customs, events and behaviours which they observed. Mass Observation and public authoring are both creative activities that seek an audience beyond their immediate community, yet at the same time they are the outputs of people attempting to record their own world and values for themselves.

In a similar way, the lifecharms, or data shells, which we created in the Lifestreams project were expressions of data manifested into material form. As such the data itself ceased to be directly readable (as it might be in a graph or chart), but was embodied in the actual form and shape of each shell. No longer intelligible to anyone but the person whose data had been the source of the shell’s growth pattern, the lifecharms signify something without revealing exactly what. As a type of self-documentation this translation of digital data collected from biosensors and life-tracking devices begins to mirror the output of our public authoring notebooks of Reite village : fascinating to outsiders, yet unintelligible if not directly explained by someone within the community that generated them. Open to interpretation, inspirational and allusive, but never didactic. Where knowledge and communication are reciprocally entangled, not inferred.

The trajectory I sketched at the beginning of this statement – linking human agency as the core of many projects I’ve led – now connects all these thoughts on the nature of what kind of future we want to build. Is it to be one where we devise systems to manage our societies that are responsive and dynamically adapt to our interactions with them, or a future where our decisions and choices are increasing defined and shaped by algorithms designed by and for the benefit of an ever more remote elite?

Engaging Patient Perspective

In 2014 I was commissioned by Professor Helen Dawes of the Movement Science Group at Oxford Brookes University to assist her team and partners in devising how to engage patient perspective in the creation of a new rehabilitation feedback tool for survivors of Traumatic Brain Injury (TBI). The project was a small part of the huge CENTER-TBI project funded by the EU and composing over 120 partners in 27 different countries.

The team wished to create a tool that would both generate useful data for clinicians and researchers as well as being useful to the survivors themselves. I was asked to help devise a strategy for engaging patient perspective and exploring its potential for the researchers who would create the tool. To a degree this was a matter of introducing the concept and some processes associated with user experience (UX) into a medical context. Whilst it might be assumed that this would be a natural fit, it was shocking to discover how alien the concept actually is of engaging patients in designing the process for monitoring and evaluating their own condition. It seems that it is a much more standard practice to ‘design out’ patients from the business of gathering objective data about their condition in case they contaminate it in some subjective way.

What I found exciting about this project was its leader’s desire to do something different, to approach the problem of gathering data about survivors’ experiences of rehabilitation in a more nuanced and engaged way, whilst still aiming to acquire ‘clean’ data that could be used by clinicians. For this to be the case we would have to negotiate a space that the tool could occupy which would satisfy the motivations of clinical researchers at the same time as offering some return that could engage the ongoing motivation of a TBI survivor to use it to regularly contribute data.

Despite the vast scale of the CENTER-TBI project as a whole (around €29m) the bulk of the funding is targeted at acute care, with around just 1% associated with rehabilitation. This project was part of that tiny fraction and consequently very limited, with my involvement being one of the smallest parts. To use this resource effectively and impart human centred design principles into a development team unfamiliar with the practice I had to devise a method of engagement that would highlight the issues and problems of not engaging survivors in the design process, so that the development team would naturally arrive at an understanding of their own; so that they would appreciate the value of actively engaging users in planning the design of the tool to avoid unnecessary pitfalls and respond to the capabilities and capacities of the people intended to both use the tool and benefit from it in some way.

The process I devised involved facilitating an initial co-discovery workshop for project team members (clinicians and researchers), followed by a co-design workshop matching team members with people with TBI at a local Headway centre in Oxford.

The first day-long workshop took place at Oxford Brooke’s in April 2014. I had invited a colleague, David Sinden, to co-facilitate the workshop with me. David is a photographer and artist, as well as an experienced media producer and curator – more specifically he has direct experience of surviving a traumatic brain injury himself, having suffered a sub-arachnoid haemorrhage in 2002. We had worked together in the early 2000s and I knew that David would bring a unique understanding not only of the issues facing a TBI survivor, but would be able to do so within the context of the needs and requirements of media production. Also part of the project team was Dr Lizzie Coles-Kemp of the Information Security Group at Royal Holloway, with whom I previously collaborated on two other engagement projects.

Over the course of the day we explored the varying motivations of the different actors (TBI survivors, clinicians, researchers etc) that would need to be considered as well as their capabilities and capacities to engage with or use the tool. We tried to scope out what the benefits of using such a tool would be for the different actors in the system, and how this could inform the design process;

I felt at the end and that we had worked towards a tool that has the potential to be useful not only after traumatic brain injury but also after any acute onset disabling condition and indeed probably in all disabling conditions. … In terms of process, I felt that the meeting was well run with an appropriate balance between allowing conversations to continue and thoughts to develop, and moving on to the next stage. [participant DW]

The workshop was subsequently documented in a bookleteer book, including post event feedback from the participants, to enable its results to be more widely shared. The book included visual annotations of the day’s discussions drawn by Proboscis associate, Mandy Tang.

Screen Shot 2015-07-30 at 13.36.33

The second workshop took place in late July at the Headway Oxford centre. I had devised a simple paper-based tool for the participants (five TBI survivors) to map their own rehabilitation journeys through a creative process, assisted by Headway volunteers and project team members. The aim of the workshop was to reveal the nature of the complexity and challenges for user engagement in the design of a rehabilitation tool by actually doing the task itself in a paper-prototyping fashion. Two of the Oxford Brooke’s development team took part and were able to experience first hand for themselves the kinds of capabilities and capacities that TBI survivors cope with.

The exercise was designed to be a simple as possible, asking the participants to just use words and simple drawing techniques to describe aspects of their rehabilitation journey. Yet not one of them felt confident enough to do the writing or drawing themselves and relied on a volunteer to help them. The group completed the tasks individually, yet openly discussed the questions and helped each other ‘remember’ things. The value of mutual support in recalling and recording individual experiences was firmly underlined. The workshop revealed to the researchers some of the considerable engagement and user experience issues which they would have to address before any tool – digital or analogue – could be used in the field.

For myself it was inspiring that the TBI survivors at the workshop found the exercise to be useful in itself, and could see the value in repeating it on a regular basis – both as a bonding exercise with other survivors and to help recognise and understand patterns in their own lives. Unfortunately I have been unable to find additional funding or resources to enable me to develop the paper-based TBI engagement tool further. I hope to find an opportunity in the not too distant future to create a version of this tool that could be used by TBI survivor groups themselves as part of their rehabilitation process, recording and reflecting on their experiences, helping each other discover or keep track of patterns that affect them.