I’ve recently re-worked some of my posts from here into publications made with bookleteer:
View other publications i’ve made on bookleteer here.
I’ve recently re-worked some of my posts from here into publications made with bookleteer:
View other publications i’ve made on bookleteer here.
In February 2014 Marina Jirotka and I met as participants at Blast Theory’s annual two day seminar, Act Otherwise. That year’s theme was “The Invisible Hand: On Profiling and Personalisation”, exploring many issues around the generation and use of “Big Data” in artworks and by artists as well as more generally in culture and society. We found ourselves sharing a healthy skepticism about the way “Big Data”, number-crunching and data visualisation are often presented as a ‘final’ and over-arching narrative to understanding modern life; as an “end to theory”. We both found this triumphalist narrative – that data-driven computation can comprehensively explain everything – to be troubling and misguided, especially as it seemed to be spreading across many other disciplines and fields of practice. The implication that both research and culture could thus be transformed into quantifiable commodities to be analysed and neatly compartmentalised purely by computational means was another major concern.
At the seminar I presented the Lifestreams data manifestation project (2012) which demonstrated how we could use our senses of touch and proportion to engage people with otherwise abstract and ephemeral information being collected about their life patterns and behaviours. The project also offered an alternative vision to the emerging “Quantified Self” and Internet of Things narratives in which complex human behaviours are often reduced to a set of data-driven variables that can be processed from sensor data. This also seemed to be an Orwellian vision that promised all kinds of benefits on the basis of a worryingly narrow perspective.
After the seminar, Marina invited me to Oxford to speak to her research group in Human Centred Computing about the Lifestreams project and my work in general. From there we began a conversation and collaboration that has continued over the past five years; most recently resulting in the UnBias Fairness Toolkit – my contribution to the two year UnBias research project (of which Marina was a Co-Investigator). We also developed two proposals that were not realised, but which coalesced some key ideas and thinking which have never-the-less flowed into other projects and activities. Both proposals revolved around ideas I was beginning at the time to crystallise – reciprocal entanglement and the data sublime. An early proposal in 2014 addressed Big Data and the Quantified Self via the data sublime, whilst the other (in 2017) focused on issues at the heart of developing Quantum Technologies. Marina’s research group is a part of the NQIT Hub, conducting studies into Responsible Research and Innovation (RRI) in the quantum field. Whilst I have since written about reciprocal entanglement in relation to quantum technologies, the ideas behind the data sublime remained unpublished in proposal documents. The purpose here is to bring them out into the light as they have a renewed relevance to the new project I am embarking on (Materialising Data, Embodying Climate Change).
The category of the sublime in literary theory and aesthetics refers to encounters with phenomena that are excessive – too much to handle – and which inspire awe or dread in the subject. What renders the experience sublime is our ability to, nevertheless, address this vastness or dreadfulness and to incorporate it into a perceptual register for meaning or sense-making. It has been a hugely important and influential category of perception in the Humanities for almost three centuries – as well as having roots in Roman-era Greek philosophy (Longinus’ On the Sublime, 1st century AD). Edmund Burke was one of the earliest English philosophers to write about it (in A Philosophical Enquiry into the Origin of Our Ideas of the Sublime and Beautiful, 1756), followed by the German philosopher Immanuel Kant (in Observations on the Feeling of the Beautiful and Sublime, 1764), then Arthur Schopenhauer, Georg Hegel, Rudolf Otto and others since (notably Jean-François Lyotard).
What the sublime offers us is a conceptual mechanism by which we can recuperate an almost overwhelming encounter with things which are too massive or complex to calculate, measure or fully comprehend. It allows us to make sense, to make meaning of an encounter with the ungraspable. It describes an ontological encounter that transforms something from being numinous (or unknowable) into something phenomenological – which we can incorporate into a narrative experience and a type of knowledge. Joseph Addison’s description in 1704, “The Alps fill the mind with an agreeable kind of horror” (from Remarks on Several Parts of Italy etc. in the years 1701, 1702, 1703), captures the internal ambiguities of the sublime – that horror can be at all agreeable – which make it such a powerful perceptual register of recuperating the excessive. Such a rupture and intertwining of perceptual and critical abilities within a person’s consciousness could, perhaps, be figured as a form of entanglement between oppositional states and phenomena – the sublime being the moment of awareness of the entanglement itself. It is, of course, always relational between the person and the thing they are encountering.
Artificial Intelligence, Machine Learning, Big Data, Algorithmic Decision-Making are the latest issues du jour, about which we are so often told that the data is too big to understand, the algorithms too complex to represent and the decision-making processes too opaque to be grasped by mere humans. Might it be possible for us to make sense and meaning of such vast quantities of data and computational processes in ways that affect our social and cultural aspirations for society beyond the purely instrumental? We could consider the speed and quantity of the data being generated, both individually and societally, as a monumental encounter. Such an encounter could then be approached as moment of the ‘data sublime’; an encounter where artistic practice may offer us alternative opportunities to assimilate and make meaning from it.
Art and aesthetics offer different ways to conceive of what happens in complex encounters than those utilised by the sciences. When you encounter a work of art, it is the experience itself which determines your aesthetic reaction to the piece. It could be one of awe, delight, revulsion or indifference – whatever it is, it is driven by similar complex factors. Each person’s own aesthetic experience is affected by the use of materials, colour, scale, lighting, sense of space and proportion as well as by their own memory, critical thought and emotion. There can be no right or wrong aesthetic experience: each person experiences a work of art in relation to the summation of their own existence.
The methodologies and critical analyses from the arts and humanities, such as aesthetics and categories like the sublime, offer alternative ways to develop new ways of realising knowledge from data and computational systems through encounters that work on multiple senses, not just via sight and sound as most contemporary technologies currently deliver it. We don’t just have to simplify and summarise data in linear ways to make it easier to represent visually on a screen (e.g. in a chart or diagram), we can also use our other senses – touch, sense of scale and balance, taste, smell, hearing, time and temperature. The data sublime in artistic encounters allows for multiple sensorial engagements, where we are reciprocally entangled in the possibility for meaning making with the work itself, through our own consciousness. It is a route away from the reductive reasoning of standardisation, quantification and calculation that lead to probabilistic and statistical interpretations. Instead it is a route to knowledge that reinserts key human qualities of judgement and imagination that can encompass the kinds of ambiguity, diversity and the unknowable that would be routinely excised from analytical systems based on quantification alone.
Evolving technologies such as Quantum computing and AI – topics of extraordinary complexity and subtlety – run counter to mundane understandings of the phenomenal world and stretch the limits of human perception. The intangible, counter-intuitive nature or sheer vastness of the science makes it hard for people to grasp, and yet so exciting in its implications for the future. The complexity and interdependence of planetary life and natural systems (such as climate) is another sphere that often seems overwhelming in terms of the scales involved. Modelling climate systems alone require some of the most complex computational methods and powerful resources. How people can make sense of such data, often geographically and experientially remote, is one of the key challenges of our age.
The MDECC project will be attempting to explore ways in which artistic expressions of data into physical manifestations (sculptures, installations and inhabitable spaces) might offer new ways for people to make sense of such remote phenomena and connect it to their own lived experiences. In this way we will be exploring the affordances of a data sublime to make climate science accessible in ways it has not been before.
This post draws together some of the many strands that have led to a new project – “Materialising Data, Embodying Climate Change” – which I have just begun with Tom Corby at Central Saint Martin’s, University of the Arts London. I want to make links across what may often seem like a wildly divergent practice – from collaborating with people in the jungle of Papua New Guinea, to working on tech ethics and policy interventions, to building experimental communications platforms, to making abstract sculptures from data, to facilitating workshops in languages I don’t speak with people who have experienced trauma I can barely comprehend.
For me there is a clear trajectory and purpose, which centres on inspiring agency in others. I believe that the answers we need to address our problems – as individuals, as communities, as peoples, as a species – are all around us, hiding in plain sight. The greatest asset we have, as human beings, is our imagination – with it we can devise solutions to whatever we encounter that holds us back, or create new problems for ourselves. Everyone is born with it, and its abundance is the true wealth of our societies and cultures. But while we have a terrible penchant for externalising imagination and corralling it in things – principally valued as commodities – yet it is ever at hand when we need it in challenging circumstances or crises.
Right now we face the implications of our species’ failure to act for a generation in the face of mounting evidence that unconstrained climate change is inevitable and will bring unfathomable disruption to all life on earth. It falls to our imaginations to grapple with these issues – to imagine different paths, to free ourselves from the restraints that have privileged some people’s imaginations and abilities as being more worthwhile or valuable than those of others. I hope with this new project not just to inform, but to inspire people to act.
Materialising Data, Embodying Climate Change is a major three year research project funded by the AHRC which will build on the legacy of the Lifestreams project which I led back in 2012 (with Stefan Kueppers), my 2014 Creativeworks Entrepreneur-in-Residence award with George Roussos at Birkbeck University of London and the experiments with data manifestation and Parkinson’s Disease which Stefan, George and I did in 2016. It also draws heavily on Tom’s long collaboration with Gavin Baily and the British Antarctic Survey creating artworks with climate data, such as Southern Ocean Studies and Northern Polar Studies. It has been a long haul to get here – I first began exploring ideas with Tom in early 2016 around our shared desire to explore how we could engage people with the complex data being generated by multiple interacting climate phenomena (e.g. sea temperature; sea salinity; polar ice extents; methane and other gas levels; krill population etc). For more than a generation, mainstream media coverage of climate change issues have rarely strayed from focusing on just a single issue – such as the ozone layer, and then CO2 levels – yet it is now clear that it is the interaction of multiple climate phenomena that are driving the changes we seek to control.
I have also had years of discussions with my friend Juan Francisco Salazar at Western Sydney University about how the Lifestreams process could engage with data about the Antarctic. Juan is a filmmaker and anthropologist who has made several trips to the Chilean Antarctic base, and made a feature-length documentary, Nightfall on Gaia, there in 2015. Our discussions have often revolved around how to make the changes that are directly perceptible in more remote and fragile ecological sites, such as Antarctica, accessible in a tangible and tactile way to people living in the urban industrial world, where experiences of the natural world have been muted by human intervention and our connection to nature is fundamentally disrupted. Juan also leads on the Antarctic Cities project, an inspiring collaboration of Australian, New Zealand and Chilean institutions linking three cities (Hobart, Christchurch and Punta Arenas) with each other and Antartica.
The data manifestation trajectory goes back way further too – to a concept of “tangible souvenirs from digital experiences” which I first formulated during the Urban Tapestries project in 2003. At the time I realised that engaging with people from a diverse set of social, educational, cultural and economic circumstances would need the creation of hybrid forms of communication and expression (both digital and physical) which would allow people to engage and participate in ways they are comfortable with. This concept further evolved over the years into experiments with sound and tactile interfaces (e.g. the “Rumbler” and “Sensographs” of Sensory Threads) as well as paper-based outputs (e.g. automatically generating StoryCubes or DIFFUSION eBooks from digital assets with the bookleteer API).
My experiences working with James Leach and the villagers of Reite in Papua New Guinea on our traditional knowledge documentation project, TKRN (2012-20), have also been of crucial importance in extending my thinking and appreciation not just of the extraordinary range of world views that exist (outside the bubble of Western, industrialised culture), but also of the kinds of knowledge that exist. Particularly those that rely on human senses beyond the visual and aural for meaning-making : such as those of touch, smell, taste, proprioception to name a few. An important insight came to me after my first visit to Reite village in 2012, not long after we had generated the initial set of “lifecharms” or data objects for Lifestreams. I wrote about this in two posts in 2013: Tactile Poetry and Digital Alchemy. James’ writing has also been an influence on my thinking, especially his essays Drum and Voice (2002), Leaving the Magic Out (2012) & The Death of a Drum (2015). My experiences in the village have also exposed me to how direct and perceptible climate change is to people who live within nature and the natural world – seasonal weather patterns do not come reliably; plants are not ready to harvest at the usual times; animals, birds and sea creatures are slowly disappearing. All this is increasing year on year, and is the daily reality for people living traditionally in the forest and on the coast.
Back in September 2017 I wrote up many of my ideas and aspirations for melding these different strands of my work in a post – Sensing Climate Change Through Empathic Encounters. Almost a year and a half has passed and it seems a good moment to revisit them in the light of my other activities – including developing the UnBias Fairness Toolkit – at the start of this new juncture. All these projects share my key aim of stimulating agency, trust and reciprocity, themes which I have also written about before (Reciprocities of Trust & Reciprocally Entangled) in different contexts, such as my collaborations with Lizzie Coles-Kemp’s Collective Securities group at Royal Holloway University of London and Marina Jirotka’s Human Centred Computing group at the University of Oxford. And lastly, but certainly not least, my long collaboration with Canadian artist and curator Andrew Hunter (since 2007 through his Render and DodoLab projects) and the creative discussions that have sustained both of us through periods of intensive work and fallow, hard times.
The following sections bring together various threads of my work. This essay is a mélange of ideas in development, weaving together the very new and those which have been delicately spun over a number of years. Something which links across all of them is ‘hiding in plain sight’ – the suggestion that what we seek is often already in front of our eyes, but occluded or hidden from our perception. Sometimes because ‘we cannot see the wood for the trees’, sometimes because an obsession with novelty and innovation blinds us to what already works well. For years I have kept a saying of Charles Eames close to my heart – “Innovate as a last resort” – and a copy of the 1953 Eames India Report close to hand. I recommend watching the wonderful film, Goods, in which Charles Eames gives a talk on the manifest joy to be found in the practical simplicity of everyday things – often overlooked but always within reach of our perception.
Three years ago, in March 2016, I was invited to present a provocation at a workshop on ethics and responsible innovation in data science hosted by the Alan Turing Institute. It was an opportunity to demonstrate the ideas behind data manifestation to a large (50+) audience of researchers from a variety of disciplines all concerned with ethics in digital innovation. My Lifestreams colleague, Stefan Kueppers, was by then beginning his PhD with George Roussos at Birkbeck and working as a research assistant on the CloudUDPRS project, devising a mobile app to enable Parkinsons patients to self-monitor their motor tests. As part of the trial, we 3D printed a series of lifecharm shells generated from 4 different patients’ data to demonstrate qualitative differences in the individual experiences of Parkinsons that were masked by scoring similarly on the Unified Parkinsons Disease Rating Scale – which is used to determine care & therapy packages. These were given out to the participants as tangible souvenirs.
My talk provoked a number of interesting conversations about the applicability of data manifestation as a means of communicating information on different levels. There were a number of privacy and cybersecurity experts present with whom I began to speculate on the potential for data manifestation (i.e. expressing data in physical forms) as a novel mode of cybersecurity – especially in the field of medical ethics and patient data. I proposed that encoding patient data in physical forms might be a useful means of allowing data to be transported and ‘shared’ with different people – such as a patient discussing their conditions with a physician. In such a situation both parties can interpret the data object and use it to have an informed conversation because they already understand the context in which the patient’s data was collected and from which the data object has been created. However, and crucially, a third party intercepting the data object would simply be unable to access the source data from which the object was generated (e.g. via reverse engineering) or be able to ascertain its context from the object itself. I refer to this approach as “data veiling” – or hiding in plain sight.
My own experience of helping family members and friends through chronic illness necessitating treatment at different hospitals and clinics, illustrated how often – and insecurely – patient data needs to be shared between multiple parties. Often this means patients having to carry paper records of their data with them to every meeting; sending paper copies by post or emailing unencrypted attachments across the internet. The opportunities for private and extremely personal data to leak out into the wider world are rife, constant and a clear threat to individuals given the intense interest in acquiring private medical data now being shown by the companies at the heart of digital technology and AI/machine learning.
Data-Veiling in this way could be an alternative means to achieve cybersecurity – allowing people to carry a physical expression of their data in a way that prevents the source data being directly accessible – the form itself would be enough for meaning to be discussed. It could allow us to discuss with others what the data implies or means by virtue of establishing tactile grammars that help us interpret the shapes and forms. It would remain secure because is unlikely that the source data could be reverse engineered from the physical form, because the (Lifestreams) method of generating the data object is not parametric but a series of structured mutations. The method flows different data streams together in the generation process which cause variations in the growth shape as they interact with each other. This is also notwithstanding additional variations introduced in the 3D printing process itself, such as surface smoothing and slight reductions in resolution depending on the type of material used to fabricate the object – such as metals, plastic, ceramic etc.
Data-veiling also has echoes with work I’ve been doing in Papua New Guinea (PNG) with indigenous people to document traditional knowledge. In such communities what Westerners think of as knowledge is understood and practiced in very different ways; knowledge is often acquired through complex rituals that make manifest personal status within the community and situate a person within a network of relationships. Having and using knowledge is a demonstration of power and ability. In the West, information generally becomes knowledge through its alienation from context into books and other forms of transactable documentation (such as films, digital files etc) that facilitate universal replicability. This is very different to cultures for whom knowledge seems to reside in how relationships are performed between people, place and things. For instance, there may be specific practices (such as magic or sorcery) which cannot be freely shared or discussed openly. Thus documentation (or expression in some kind of object through design) might serve – sufficiently – as a signpost for those who do have the knowledge; it could indicate the lineage that their knowledge has been acquired through and, how transmission to others might occur (through some form of ritual exchange).
In the village I visit in PNG (when referring to practices and phenomena which Westerners would call magic) people slip from speaking in Tok Pisin (the common national language) into their own local language (Nekgini – spoken by less than a thousand locals) and into ‘hap-tok’ (“half speech”) – a kind of allusive mode of speaking around a subject without discussing it directly. The ability to participate in and understand the meaning behind such discussions would demonstrate either knowledge itself, and the ability to understand how to ‘read’ the signposts being referred to, or to the constraints under which such knowledge could be acquired from those who have it. Porer Nombo, one of the village elders who James has worked with for over two decades, suggested something along these lines when giving a presentation about the ethno-botanical book he co-wrote with James, Reite Plants (as reported in James’ essay, Leaving the Magic Out). To me, this is another kind of hiding in plain sight – documenting part of a story and including clues for the willing to discover how further layers might be accessed. Revealing just enough, sufficient for others to make use of what is spoken.
By encoding sensitive data into physical abstractions it might be possible to conduct informed discussions about the real world meaning of datasets within specific contexts – without having to access or share the data itself all the time. Data veiling could offer a form of signposting or symbolically representing sufficient data without having to reveal its full detail. Shifting our focus from data to the patterns and meanings we can interpret from it.
Over the past two and a half years my attention has been largely taken up with my role in the UnBias project, and in particular, devising practical and pragmatic ways to foster both an awareness of bias, trust and fairness in algorithmic systems, and how to “do ethics”. ‘AI Ethics’ went mainstream in 2018 and there is now huge interest in how those companies which build digital technologies and services (especially those involved in AI and machine learning) can deliver responsible innovation through ethical design and development processes.
The egregious harms to democracy, as well as to individuals, which have been exposed in the past few years (Snowden, Cambridge Analytica etc) give this a tremendous urgency. More and more systems and decisions seem to be being automated, with all the damaging effects long-predicted becoming day-to-day reality. Whilst human consciousness is able to deal with contradictory states and can make informed judgements that reflect the complexity of contexts and situations, we are yet to devise machines that can handle such complexity themselves. Perhaps this is an inherent weakness of the binary mathematics they are built on? Increasingly the evidence emerges not only of systemic bias being found in automated systems (mostly trained on inherently biased datasets), but also of how such systems are amplifying pre-existing biases and prejudicial outcomes which have the greatest impact on the most vulnerable in society. I fear that the kinds of automation offered by AI/Machine learning, based essentially on inferences drawn from incomplete and prejudicial datasets, might only ever be a zero-sum game that inevitably leads to a “statistical… regression to the mean” (quoted from Alan Blackwell, below).
The danger is not the creation of systems that become maliciously intelligent, but of systems that are designed to be inhumane through neglect of the individual, social and political consequences of technical decisions.
Alan Blackwell, “Interacting with an Inferred World: The Challenge of Machine Learning for Humane Computer Interaction“
The mainstream narrative for increasing automation has for a long time revolved around “efficiency” – that machines are more efficient than people, they cost less, are somehow ‘neutral’ etc etc. We need to closely interrogate such narratives and expose their fallacy for what it is: a mask to cover the reality of where that vast bulk of the benefits flowing from such automation go. This is hardly a new situation – for hundreds of years people whose ways of life and agency have been undermined and eroded by capital and automation have tried to resist what is essentially a subjugation. Their descendants created unions, cooperatives and formed mutual societies to win back some of their lost agency and initiative, but history shows us it is a recurring theme, for which almost each generation has to find new answers and approaches.
We are also beginning to see the end of another narrative that has specifically woven itself around digital technologies – that innovation should be unfettered by regulation, and that the digital tech industries are creating change faster than laws and regulations can keep up with them. We live in a time where a corporatist agenda seems to have gained a triumphal ascendancy, and where mantras like “information wants to be free” are posed against a picture of a heavy-handed state endlessly creating red-tape to frustrate the innovators and entrepreneurs who are cast as the only ones who create new value in society. Such narratives are as hollow and self-serving as those which reduce all values to the purely monetary and refuse to account for key human values such as kindness, love, generosity, happiness, play, pleasure and joy.
Returning the the theme of hiding in plain sight, I have drawn great inspiration from the work of William Perrin and Lorna Woods on the concept of duty of care in social media regulation for the Carnegie UK Trust. They look back to the groundbreaking UK Health And Safety legislation of 1974 which harmonised protections for workers across all industries and workplaces under a general duty of care, and extended those duties to pro-active harm anticipation and reduction through due diligence to the rest of society. Their suggestion is to build a new regulatory framework which draws upon the well-established principles of Health and Safety legislation to create a proactive duty of care on providers of digital services and platforms for harm reduction. This would shift the emphasis from the current post hoc situation where liability is assessed and punished only after harm has been experienced, to an a priori approach based on taking care as a first principle. A fundamentally diametric approach to the infamous Silicon Valley mantra of “move fast and break things”. Digital Safety rather than Digital Security – proactive efforts that care for rather than seek redress for harms already done.
What I like so much about this approach is that it builds trust based on reciprocity – when we know that people are exercising a duty of care on all our behalfs, we trust that they will do their best to not harm others. Trust is both formed through, and builds on, relationships and the performance of the values that underpin them, not simply through transactions. For this reason I remain highly skeptical of the utopian claims for ‘smart contracts‘ that have surrounded the blockchain hype in recent years. Whilst they may offer some benefits in terms of automating certain types of exchange, I see specific dangers in attempting to reduce all kinds of complex relationships and exchanges to contractual standards based on transactions. What of trust? What of reciprocal obligations and fundamental rights? There is always a tension between human rights and contract law – an asymmetry where the weaker party is often induced to sign away their rights for incommensurate benefits to the advantage of the stronger party. Unless we have robust institutions and frameworks for the protection of the vulnerable, and people have access to education and information about their rights and values, we will forever be at the mercy of those who corral power and wealth for themselves, and seek to buy their way out of obligations to others through unbalanced contracts and negotiating power.
Who gains from a world where we are increasingly encompassed by contracts that privilege transactional relationships? In which social domain are such contracts disputed and settled? Traditionally lawyers and those who can afford to be litigious have been the gatekeepers and prime movers in contractual situations. Smart contracts may seek to subvert this nexus, but will they really democratise these processes or simply alter the landscape of who holds the upper hand in favour of the programmers – and their employers – who are able to understand and manipulate the arcane new languages that evolve with them?
It seems to me that human rights and concepts like duty of care and the precautionary principle are a critical baseline that are collective in nature – i.e they support everyone in society regardless of status or situation. In contrast to this are contracts and transactional systems which inevitably privilege negotiations for those already invested with power and resources. If we truly want a fair and just society, supported by tools and technologies that we trust, then it seems to me that we must reinvigorate constitutional democracy with transparent rights and laws, as well as accessible education and information about what they mean and how they affect people. That way we can take care of each other.
Despite the focus of my new project on climate change, I’ve barely touched on it here and it is, to my mind, the most critical issue we face as a species living in a fragile ecology. My preoccupation with agency is, though, at the very heart of how I think we can inspire and stimulate the kinds of societal change necessary to preserve as much of life on earth as we can. I am neither an ecologist nor a climate activist, but I believe that each person must find within their own skillset and area of work and life the means to contribute positively to such change. Not just in terms of our personal habits of consumption, but in the effect that our work can have on others and the processes we live within. As an artist and designer, I feel it is my duty to achieve this the best way I can – by inspiring others and, ideally, stimulating them to be hopeful and to take agency for themselves.
I have also been inspired by coming across a remarkable initiative that took place in the UK in the 1970s – the Lucas Plan. Faced with the threat of massive redundancies the shop stewards of several unions representing workers at Lucas Aerospace formed a ‘Combine’ to develop an alternative business and industrial plan for the company. This was based on producing ‘socially useful products’ (at the time 50% of Lucas’ products were parts for military hardware) and around 150 such products were proposed by the workers, based on existing skills, tooling and capabilities. Ultimately they were dismissed by the management board. Never-the-less, the initiative inspired many other outcomes and initiatives – some of which continue to this day. (Incidentally, many of the then experimental products are now mainstream). Watch a trailer for Steve Sprung’s forthcoming documentary, The Plan, or an original 1978 documentary by the Open University/BBC. To me this is an example of what happens when people begin to take agency for themselves and reimagine some of the basic premises about what it means to be responsible for their actions – for the things they produce as a worker in a firm or institution, for the impact this has on society and the environment. With this sort of thinking and action we might yet reconfigure our society and industry towards a trajectory that minimises the impact on the ecosystem and life in general, and mitigates some of the worst excesses of the past for a sustainable future.
One of the key figures in the Combine was Mike Cooley, whose 1980 book, Architect or Bee? The Human Price of Technology, sets out a clear vision for human-centred, socially useful design and production. His thoughts on ‘augmented and symbiotic systems’ in favour of automation or classical AI systems are particularly brilliant. He proposes that such technologies be created to enhance human capabilities, not replace them. He describes the path to expertise as being one which increasingly expands on the basic rules acquired along the learning journey adding intuition, insight and imagination as one’s experience grows. The goal he suggests is for technologies to enable and augment everyone’s capacities, not to de-skill people merely to increase the profit share for managers and owners. He calls for humane technologies that not only increase our abilities to make things, but to appreciate life, freedom and choice as well. In one example, Cooley writes of designing expert medical systems in the 1980s which,
“provide an interaction between the ‘facts of the domain’ and the fuzzy reasoning, tacit knowledge, imagination and heuristics of the expert, and no attempt is made to reduce all these aspects to a rule-based system – the system is seen as something that aids rather than replaces the expert.”
Although not long, the scope of this book is vast and is full of insights into the operation of power in society, politics, business and industry as well as offering practical suggestions for how to address it creatively and with intelligence. It is a vision for empowering people to act for mutual benefit, in sympathy with the ecosystem which sustains us. As we grapple with the choices ahead of us for democracy and ecological sustainability, we could do worse than take a few leaves from this book:
“The choices are essentially political and ideological rather than technological. As we design technological systems, we are in fact designing a set of social relationships, and as we question those social relationships and attempt to design systems differently, we are then beginning to challenge, in a fundamental way, power structures in society.”
My hope for the Materialising Data project is that we can find new ways to create empathic encounters with the urgent, complex reality of climate change for people to whom it is not yet as directly appreciable as it is for those living closer to nature. In challenging the dominant modes and forms of how we make meaning from data, I hope that we too can begin to challenge some of the power structures in society that have remained inert in the face of growing emergency for almost my whole lifetime. We have a duty of care for future generations, and for all the other forms of life we share this planet with. The answers remain before us, hiding in plain sight – we just have to be brave enough to adopt them.
Despair is not an option:
Anticipate the worst;
Hope for the best;
Cope with what comes;
Tread lightly on the earth.
London, February 2019
This is a slightly tweaked version of a presentation I gave recently for a Fellowship application. My proposal was to use the Fellowship to advance the work I have started this year in adapting the processes of data manifestation developed in the Lifestreams project in creating tangible artefacts that communicate the complexly interconnected phenomena of climate research. And to also weave into this narrative approaches inspired by other cultures (especially non-Western “indigenous” peoples) who have developed other ways of recording and sharing knowledge of change over long timespans.
I see art as a way of life, a way of being, rather than the things I make. My practice is founded on bringing the sensibility of being an artist into direct contact with the world through the collaborations I undertake in different contexts and places and with the different communities which I engage with.
As an artist my goal is not to put artworks into galleries, but to weave art into everyday life – not as an exception or decoration, but as a vital and transformative process.
The way I do this is to create the conditions for stories to emerge from the material and interactions of these collaborations, and to help communicate the values expressed by the participants to others: storymaking instead of storytelling.
My interests are transdisciplinary – exploring the intersections of Art, Design and Science and two key strands in my work are Data Manifestation and Public Authoring. These two strands delve into the heart of how artistic practice can assist us in making sense of and defining values about our complex selves and environments: through sensory engagement and knowledge creation, documentation and dissemination.
My proposal focuses on climate change – and specifically on exploring alternative ways to allow people to appreciate and make sense of the complex interdependent interactions between different climate phenomena that are more often isolated and presented independently of one another. Mainstream debates over the past 30 years have tended to treat ‘climate change’ as a series of individual issues – such as the ozone layer, carbon dioxide emissions, global warming, sea ice reduction and glacial melt. In parallel the measures to ‘deal’ with them are often presented as separate strategies too. Whilst we know, in the abstract, that we live within a single global ecosystem, it is passingly rare to see these issues presented in ways that help us understand and relate to their interdependencies.
I propose to use data manifestation and public authoring as experimental means to generate opportunities for people to experience empathic encounters that go beyond the instrumentality of two dimensional graphs and visualisations. In particular I hope to explore more spatial and architectural expressions which can trigger a range of human senses in making meaning from data about climate change.
These are what I call “reciprocal entanglements” where people encountering the work perceive themselves in a direct relationship with it and with meaning making, thus transforming their understanding of issues that are at the heart of humanity’s future.
Over the past few years I have been working with indigenous people in Papua New Guinea and Vanuatu on adapting, for their situation and purposes, some of the public authoring tools and techniques I’ve developed.
This has been a mutual exchange of cultural practices intended to reinvigorate intergenerational sharing of traditional knowledge. They hope to preserve their traditional and dynamic way of life from being overwhelmed by the changes wrought by money and industrialised resource extraction. In our terms, they seek to retain their culture’s resilience and sustainable relationship to their specific environment and not be subsumed into our Western worldview. This relationship has given me new perspectives on how environmental change is experienced, understood and shared – where knowledge of such things is inherently local. It is deeply and necessarily intertwined with the daily practices that are required for subsistence, as well as with the community memories stretching back many generations.
My aim is to investigate different ways of making the complex data about climate change tangible for people in post-industrialised societies who do not have the kind of direct and proximate connection to land, sea and the natural world that their ancestors had or that those communities have who still live in rural, often pre-industrialised settings.
Recently I have been inspired by reports on comparative studies of Aboriginal stories and traditions in Australia with scientific evidence of environmental change. These studies have exposed remarkable accuracy in the stories (to within one generation of accuracy) over very long timespans when describing phenomena such as sea level changes. I believe these stories and traditions persist in such cultures because they are neither remote nor alienated from context. They remain proximate and directly relational to the people who have lived with and through them.
How then, might it be possible to devise ways to express environmental change across time for urbanised people in the post-industrialised world – like ourselves – who do not experience such proximity on an everyday basis? And whose life choices will increasingly be made within the context of momentous changes in how we live our lives?
I see my engagements with climate scientists, social and cultural geographers and others doing research and other activities as a form of reciprocal entanglement too. Using tools I’ve developed for public authoring, such as the bookleteer formats, I propose to conduct an auto-ethnography of this process. By sharing my practice and experience of artistic fieldwork in an open and collaborative manner, I hope to stimulate discussion and debate, as well as to invite others to participate. I hope to weave together some of my experiences of working with indigenous people and their insights, with the observations and data of climate science to help us understand the effects and impacts of our actions, to to help us think about what our future choices might be.
Earlier this year I was shortlisted for a residency in Lincoln as part of a project celebrating the 800th anniversary of the Magna Carta. My proposal (“Lincoln Debates“) was to work with young people in the city to re-evaluate the state of our society and to bring their voices concretely in to discussions of where we go next. I proposed to engage people in gathering and sharing their voices – through collaborative public authoring and the making of material artefacts, much as I do in my work with indigenous communities in Papua New Guinea and my work with data manifestation. Embedding their concerns and aspirations for civic empowerment within the fabric of the city through events and with physical, digital and other media I hoped to devise a project that could resonate for future generations and inspire in them a continual engagement with power, rights, obligations and responsibilities.
1647 Putney Debates
My project title alluded specifically to the debates held in October 1647 at St Mary’s Church, Putney between the representatives (“Agitators/New Agents”) of the rank and file of the New Model Army, and the Army Grandees (Oliver Cromwell, Henry Ireton etc). The debates were part of a proposed settlement towards the end of the English Civil Wars to determine a new future for England (and by implication Scotland, Wales and Ireland). The men of the army had developed their own manifesto, which would later become a manifesto for a proto-socially democratic constitution for England – An Agreement of the Free People of England. Their demands were essentially communitarian and advocated for much of what we would now recognise as the basis of a functioning democracy, but where there would be no inherited privilege (thus ‘levelling’ each person’s status). The Debates were ultimately unsuccessful, but the discussions that took place and the manifestos that resulted became the inspiration for future milestones in our own and other country’s path to freedom and democracy.
Modern Putney Debates
As we lurch forwards into a new time of uncertainty and division in this country, it seems to me that we could do with more such debates and proposals – not merely as an exercise in creative agency, but vitally as a positive beacon for the future of our country, or countries, our identity and our aspirations to do good in the world.
What would it take to organise some events that draw inspiration from the Levellers of 1647 and strive to propose a new social contract founded on equity, compassion, responsibility, empathy and agency?
Last year I published a series of books that brought together key texts that derive from Magna Carta and the Charter of the Forest – spanning 900 years they represent a stunning legacy of hope against oppression, subjugation and exploitation. Now is the time to start adding to that legacy with something new.
Video of my talk at the Open Data Institute on Friday 17th June 2016:
For further details, read my post, How Do We Know?
The following is the text of a ‘provocation’ I gave at the Responsible, Ethics-Aware Research and Innovation in Data Science symposium hosted by the Alan Turing Institute in March 2016. For the event, my colleague Stefan Kueppers and Professor George Roussos at Birkbeck University of London generated a new set of Lifecharm shells using data from a research trial with sufferers of Parkinsons Disease. We 3D printed multiple copies of 4 shells which were given to the participants as a tangible souvenir of the data manifestation concept.
I’m going to speak today about a concept I call “data manifestation” and how I believe it can add a significant dimension to data science and how we make meaning from data.
First I wish to pose a question –
How do we know what we know?
And how do we make meaning from what we know?
The answer is, of course, through the interaction of our senses with the stimuli coming in to them from the external world – and how this is processed by our consciousness and woven in together with our memories and emotions.
This is a system of high complexity and entanglement. It produces judgements based on multiple factors and dimensions all working together.
I believe that this is qualitatively different to the kinds of inference driven systems that are increasingly coming to dominate our civilisation and how it operates.
For instance: when we feel that something we have eaten is delicious, how do we know this?
It is not just the taste that defines this judgement, but includes many other factors such as smell, look, texture, sound, shape and size, whether it invokes memories, or causes a momentary shift in our sense of balance. If each of these factors is seen as a data input, how would we begin to map such complexity into the decision-making systems of today?
Art and aesthetics are another example of this complexity. When you encounter a work of art the experience itself which determines your aesthetic reaction to the piece – whether one of awe, delight, revulsion or indifference – is driven by similar complex factors. The use of materials, colour, scale, lighting, etc as well as memory and emotion create each person’s own aesthetic experience. There can be no right or wrong aesthetic experience, only how each individual experiences a work of art in relation to the summation of their own existence.
Much of the data that is captured, recorded, analysed and used as the basis of inferential decision making systems in the everyday world is principally derived from visual interfaces, from the writing of the software that generates it to the representation of the data in screen-based visualisations – from spreadsheets, to graphs and animations. There are in a few cases some sonic representations of data, and in extremely rare cases some haptic interfaces. But on the whole the way we create, analyse and present data is visual and screen-based.
But not all experiences are visual. And my proposition is that by failing to encompass the whole and extraordinary sensitivity and careful tuning of the human sensorium in our methods of expressing data we are missing huge potential.
Back in 2012 Proboscis was commissioned to collaborate with scientists at Philips Research Lab in Cambridge as part of a public art programme. We were asked to explore the problem posed by the failure of commercial biosensor devices to engage people in leading more healthy lifestyles. Essentially the FitBits, Fuelbands and similar devices were being switched off within a few weeks of activation. Philips is a big player in the separate business of TeleHealth (where people with serious medical conditions have instrumented homes that record and transmit data to health professionals and back to the users). They were interested in how these emerging life-tracker devices could have positive well-being and health benefits for nominally healthy people over the long term. But the usage data suggested this was simply not happening.
Looking at the problem, our insight as artists was that this was an issue of relevance to people’s sense of their own self, to their identity. Our hunch was that the ways in which the narrative of a person’s life helps construct their sense of self were just not being engaged by the graphs of data represented on smartphone screens or laptops. Humans frequently invest meaning in objects that act as triggers for memory and personal significance. These can be highly ritualised objects such as wedding rings or other kinds of jewellery. They can also be seemingly random objects, such as a piece of cloth or a pebble from a beach. With all these things, the memory is evoked not just by the sight of the object, but through other factors such as the way it feels.
Our insight was to connect this tactile relationship to meaning with the kinds of data that could be collected by tracking devices. We built a simple data logger with off the shelf components and collected a range of data over a week from members of our studio: step count; pulse rate, sleep patterns, blood pressure, stress factors etc.
My colleague Stefan Kueppers then developed an algorithm based process to flow this data into a shape that could be 3D printed. We looked to nature for inspiration and realised that the mathematics and geometries in shells would allow for extraordinarily complex and unique expressions of data in a 3 dimensional form. Here are a few examples of the objects we made from our own biodata. We called these “Lifecharms” and the project, “Lifestreams”.
What I am going to show you next is our latest experiment. We have been collaborating with Professor George Roussos at Birkbeck (where Stefan is currently working on his PhD about the Lifestreams work) to adapt and extend the data manifestation concept to other uses.
Birkbeck have been running a data collection project working with Parkinson’s Disease patients – mapping the wide variation of symptoms that they experience. Parkinsons is a disease encompassing high variability in symptoms between individuals; ideally this ought to imply a high degree of personalisation in their medical care and disease management regimes. Yet health policy and medical care management for Parkinson’s sufferers are determined by a ‘scale’ (called the UPDRS) derived from about 70 factors of motor performance criteria a Parkinsons patient is classified against. These factors are collapsed into a single “summary statistic”, which is then used to assign treatment for patients according to where they lie on the scale. Whilst this is an effective way to communicate the multiple dimensions of Parkinson’s on a linear scale of progression, because of the enormous variation it is common for people with entirely different symptoms to score similarly on the scale, yet require very different care.
For patients and health professionals alike, finding ways to express the unique characteristics of an individual’s actual experience of Parkinson’s would be a big step both for personal dignity and as a way to convey the wide variations in an easily appreciable manner to policymakers.
What you see in these four examples here, a version of which you are being given to take away and keep, is an expression of three data sources from four individuals who participated in the Parkinson’s trial, each mapped in 3 dimensions into the form of a shell. They demonstrate the clear variations in symptoms experienced by these individuals. The data used were :
1) Tremor in the Left Hand – which maps to how much the spiral of the shell stretches outwards.
2) Left Leg Agility – which maps to the overall growth scale of the shell (i.e. how big it gets in terms of volume)
3) Two Target Fingertapping (Left Hand) – which maps to the change in the frequency of the recurring ridges on the exterior of the shell.
These four shells are only a very initial experiment and there are many more factors that we will be able to control with data sources from the trial in future iterations, such as the number and increment of rotations, ripples in the curve of the shell exterior, the sweep curve of the main shell, twists in the shell curve, other geometric entities occuring on the shell surface (spines and nodules) as well as the number of segments that make up the whole shell.
A key indicator in Parkinsons is how it affects a person’s left and right sides of their body so, in the future, we also propose to develop twin or “clamshell” forms that spiral out from a common centre and reflect differences recorded between a person’s left and right symmetries.
As you can see, and soon feel, even with just 3 data sources informing the shell growth parameters, it is possible to appreciate the individuality of each person in a way that is qualitatively different to looking at a graph. As we introduce more granularity and complexity, the individuation will become even more pronounced.
Taken as snapshots, such shells, could over time reflect the changes in Parkinson’s symptoms as experienced by sufferers.
What use is this? We believe that this kind of approach offers a very different and rich way to express complex datasets for making meaning in a world where data is ubiquitous, is increasing exponentially and – as we are constantly being told – is overwhelming us.
My proposition is that in this way we can find alternative ways to make meaning and take decisions based on human insight and judgement from complex data sets. We don’t just have to simplify and summarise data in linear ways to make it easier to represent visually on a screen, we can also use our other senses – touch, sense of scale and balance, taste, smell, hearing, time and temperature. We can also benefit from other knowledge traditions (as well as contemporary science) and apply methodologies and critical analysis from the arts and humanities, such as aesthetics, to make meaning and draw judgements from highly detailed material artefacts that reify complex data sets into actual things, not just representations on a screen.
Characteristics of the Shells
Shell 1 – spiral not so pronounced, which indicates that the left hand tremor amplitude is not so strong; the shell size is quite big indicating the patient has significant increase in amplitude of left leg tremor; the ridges are also become quite pronounced indicating that there is a growng lack of accuracy and slowness in finger tapping.
Shell 2 – the spiral is at the higher end, suggesting the patient has a higher amplitude of left hand tremor; the shell size is a bit smaller so the patient experiences less left leg tremor amplitude; the ridges are very small indicating a higher degree of accuracy and speed in the patient’s finger tapping.
Shell 3 – the spiral is a median, indicating a moderate but not pronounced amplitude of left hand tremor; the shell size is slighty above average, suggesting increasing tremor in the left leg; and the ridges are fairly pronounced suggesting a greater lack of accuracy and slowness in finger tapping.
Shell 4 – very pronounced spiral suggesting very high amplitude of left hand tremor; shell size is smaller indicating less amplitude of left leg tremor; the ridges are very pronounced suggesting a high lack of accuracy and slowness in finger tapping.
This statement of my current thinking on the topics of agency, data and quantum computing was originally presented to the Human Centred Computing research group at the University of Oxford on October 1st 2015. The group, with whom I have been developing a dialogue since Spring 2014, is led by Professor Marina Jirotka and has a particular focus on Responsible Research and Innovation. They are involved in embedding RRI into NQIT – the Networked Quantum Information Technologies Hub based at Oxford. I have made a few changes to the original text inspired by the group’s extremely thoughtful comments and discussion.
At the core of all my work and projects over the past 20 years is the theme of agency. By that I mean our ability as humans to act on our own initiative, to make informed decisions and choices. To be willing actors rather than directed subjects in all areas of life. In my experience open and transparent communications are fundamental to such an aspiration, and many of my projects explore ways and means of enabling people to communicate and interact with other people and systems to achieve this.
Looking back, there is a clear trajectory in my work which emphasises that systems should engage people and empower them, not seek to condition their experiences or shape their lives. From my work on the convergence of mobile phones with wireless internet and GIS mapping technologies in the early 2000s which has been described as a kind of proto-social media (Urban Tapestries), to experiments with sensors and platforms for citizen science style pollution mapping (Feral Robots & Snout), as well as experiments in data manifestation as a critique of the quantified self meme (Lifestreams). In my opinion, too often the technologies that are deployed across society are not intended to benefit all, but are contingent on bringing benefit to a privileged few – those who build them and those who commission and own the systems being built.
I believe that human individualities and different people’s idiosyncracies of learning and understanding are crucial social assets that enrich our collective experiences of life. I want to explore how we can design systems that adapt to these values and incorporate their dynamic into their very fabric. I want to find alternatives to the expanding deployment of systems and technologies that shape and manage behaviour by imposing rigid and inflexible decisions or choices. I believe that we need to question and challenge the mindset that sees the efficiency of algorithms and data-driven inference as the pinnacle of how human societies should be run.
One of the ways I think we can begin to revolutionise our relationship to data is by bridging the biological and digital. If we can design ways to utilise the whole human sensorium (not just vision and hearing) for sense-making and interpretation, then I believe that we can make complex information tangible and appreciable in richer and more nuanced ways. This means departing fundamentally from normative data representations on computer screens. It means embodying them in reciprocally interactive engagements that afford us greater use of our highly developed senses – what I have been calling “data manifestation”. This could allow us to experience data in ways that reveal things we have hitherto not considered possible. It may also reveal contingencies and limitations in what kinds of data are being collected – and may lead us to collect different kinds of data that have perhaps been overlooked.
How might we enliven relations between humans and machines so that they can be mutually influential rather than unbalanced in favour of one side or the other? I suspect that the promise of quantum computing, with its multiple states, may offer something along these lines, but only if we, as researchers and designers, have the courage of our imagination to make such a future possible.
Entangled Engagement & Quantum Computing
I have been thinking recently about the nature of entanglement – as far as I can understand it as a lay person not a scientist – and how the highly metaphoric language itself suggests an unintentional or, at least, ambiguous state. If we think about how we use the word ‘entangled’ in everyday situations it is to describe something, or some things, that have become entangled accidentally – without the deliberate decision to do so.
I have titled this statement “reciprocally entangled” because I think that the promise of quantum computing is one we must have agency with and choose to engage with, rather than unwittingly becoming caught within. That we might deliberately choose to enmesh ourselves with a system that, like humans, can be more than a cascade of rules and simple on/off decisions, would be a significant revolution in how we decide to run our societies.
I think that this idea of ‘unwitting entanglement’ characterises many of the fundamental problems we are experiencing with the so-called Big Data revolution, where we have suddenly begun to find ourselves subject to systems that are ruled by inference and not by balanced judgement. My proposition is that just as we might think that our society and civilisation has become enmeshed in a complex set of interlocking inference systems which define people’s lives in ever more intrusive ways, so might we begin to think of quantum computing – and being reciprocally entangled with such a system – as being closer to the kind of complex and consciousness-driven judgement that previously defined our choices and decisions (since, say, the Enlightenment).
More and more, the effects of decisions made by data-driven inference are coming to pervade aspects of our everyday lives. It is not always clear how such systemic decisions are arrived at, but it is undeniable that people are both shaping and having their choices shaped as an affect of the increasing reliance on so-called Big Data systems. A kind of blind self-governance that could so easily tip over into forms of self-censorship, self-privation and self-denial.
Such choices are most visible to us on an everyday level in the recommendations we see in internet shopping (“you’ve bought this and might like that”) and what social media systems select to populate our profile feeds (both adverts and posts by our connections). Their use in electronic financial services is widely known of, but little known about – obscured behind a veil of exclusivity, secrecy and the disparities of wealth and power. The role of such systems in health is also increasing, driven as much by attitudes towards risk and liability in the health insurance industry as by advances in medicine, wellbeing and disease prevention. These opaque applications remain worrying precisely because they flow against the transparencies of fairness and democracy that our society has been implementing over the past few centuries.
But what if we can rethink how we interact with systems as reciprocal engagements? And what if we were to see them as entangled relationships at the same time?
Perception is more than simply seeing
How humans perceive and create meaning is an associative process that is fundamentally different to inference from data. It is an expression of the difference between consciousness and a structured system; and, furthermore, it is more than what is offered by systems that mimic ‘neural nets’. Aesthetics and how we make meaning from artworks gives us a concrete example of how this operates in practice. Aesthetics cannot be taught, but flows from a dynamic interweaving or perhaps an entanglement of our memories and experiences with our physical senses. At each moment of experiencing something in an encounter with an artwork – writing, painting, sculpture, music or performance – we are existing in an entangled moment blending the now with the summation of our conscious sense of self. We are not inferring meaning (as a structured data-driven system might), but actually making meaning from the experience itself.
So-called neuro-aesthetics has sought to find a physical explanation of this process in the chemical reactions in the brain. But so far it has not been able to,
“ Objects are not triggers for internal events in the nervous system; they are opportunities or affordances for our continuing transactions with them. …
Art is experienced in the setting of argument, criticism, and persuasion. This is all compatible, Kant realized, with the fact that there is no way of adjudicating disputes in this area, that there are no decision procedures, no rules, no way of proving who’s right and wrong.”
Alva Noë, How Art Reveals the Limits of Neuroscience, The Chronicle Review
Beyond Visualisation : Embodied Entanglements
What if we can bridge the digital and biological by utilising the whole human sensorium for sense-making and interpretation – going beyond just vision and hearing? What if we could make complex data tangible and appreciable to a range of human senses? Embodied and felt instead of just seen and heard. In turn the experiences could be fed back in to data systems as a new set of parameters that could adjust the nature of systems that use the data. This was the theme of my Creativeworks residency with the Computer Science department at Birkbeck in Autumn 2014. The workshop I devised to help communicate the potential of multisensory expressions of data has been a crucial step in continuing to develop the ideas begun with the Lifestreams collaboration with Philips Research in 2012.
What I hope to do next is to embark on a journey of collaboration and discovery to demonstrate the potential for social change that could be unleashed by developing multisensory interactions with the digital data that is increasingly measuring, being analysed and governing our daily lives. Humans have extraordinary sensory capabilities which are not currently being used in how we encounter data – principally through screens (sight), occasionally sound and, rarely, through haptics (touch). I believe that this leaves us impoverished. By expressing digital data in new forms we could unlock entirely new modalities for recording, sharing and understanding how we live our lives : from experiences of illness and rehabilitation via biosensors to how we make sense of the Big Data that now shape and govern our society.
I hope to explore this theme along a series of trajectories and to demonstrate – to scientists, technologists, designers and policymakers and to the wider public beyond – how we could create transformational ways for people to engage with and make sense of data. I aim to work collaboratively (with old and new partners) to develop projects and experiments that express digital data in different ways that engage human biological senses other than just sight and hearing – such as touch, smell, taste, balance, temperature, proprioception and time etc. I am particularly keen to engage with researchers in the biological and life sciences (and hybrid fields such as computational biology) to create a bridge between how people interact with and make sense of the biological world around us, and with digital systems.
Documenting Traditional Knowledge in Papua New Guinea
Alongside these concerns I am working with Professor James Leach and a community of indigenous people (from Reite village) living a traditional way of life in the jungle in Madang Province, Papua New Guinea. There we are co-developing a simple hybrid digital/physical toolkit for people to self-document local traditional knowledge of plants, customs, design and productions techniques (TK Reite Notebooks). The tools and techniques we are co-designing there are based on my pioneering work since the late 1990s in hybrid digital/physical publishing. It uses the Diffusion eBook format, a unique paper folding method invented in 1999-2000, as well as the bookleteer.com self-publishing platform which was created and launched in 2009. This is another form of what I have called “public authoring” – making it possible for people to communicate things of value in open and shareable ways.
What is clear is that Reite people’s culture doesn’t consider knowledge as a series of static objects – in the Cartesian tradition of Western Knowledge – but as relational. Knowledge is constructed and exists in the relations between humans that hold it, share it and pass it on to future generations. It is always contextual and situational. To me, this establishes a conceptual link with what I have been exploring in my data manifestation work – that how we develop understandings of information and the ways it can be encoded in forms or media is located in the experience of the encounter, and our conscious ability to reweave our experiences in the present with those in our memories of the past.
One of the most striking things about this project is the difference between traditional anthropological ethnography and self-documentation that emerges from its results. What people choose to value and document may not necessarily be intelligible to outsiders, or of any interest. This flows against the mainstream of anthropology and ethnography which seeks to observe, understand and explain a culture to others. I feel that my conception of public authoring is akin to the aims of the originators of Mass Observation, who proposed an “anthropology of ourselves” back in the late 1930s. They thought that it was important for ordinary people to document ordinary life, and developed a framework that encouraged people from all walks of life to record and submit reports on things, customs, events and behaviours which they observed. Mass Observation and public authoring are both creative activities that seek an audience beyond their immediate community, yet at the same time they are the outputs of people attempting to record their own world and values for themselves.
In a similar way, the lifecharms, or data shells, which we created in the Lifestreams project were expressions of data manifested into material form. As such the data itself ceased to be directly readable (as it might be in a graph or chart), but was embodied in the actual form and shape of each shell. No longer intelligible to anyone but the person whose data had been the source of the shell’s growth pattern, the lifecharms signify something without revealing exactly what. As a type of self-documentation this translation of digital data collected from biosensors and life-tracking devices begins to mirror the output of our public authoring notebooks of Reite village : fascinating to outsiders, yet unintelligible if not directly explained by someone within the community that generated them. Open to interpretation, inspirational and allusive, but never didactic. Where knowledge and communication are reciprocally entangled, not inferred.
The trajectory I sketched at the beginning of this statement – linking human agency as the core of many projects I’ve led – now connects all these thoughts on the nature of what kind of future we want to build. Is it to be one where we devise systems to manage our societies that are responsive and dynamically adapt to our interactions with them, or a future where our decisions and choices are increasing defined and shaped by algorithms designed by and for the benefit of an ever more remote elite?
Last week I was in Edinburgh to run my co-discovery Embodying Data Workshop with 24 of Chris Speed‘s Design Informatics Masters students at the Art College. The workshop was devised a few months ago as part of my Creativeworks residency at Birkbeck College to introduce computer scientists to the possibilities of approaching the problem of data analysis and computation differently by manifesting data in tangible ways. Thus we may bring more of our human senses to bear on meaning making than merely relying, almost exclusively, on vision and hearing as with standard data visualisation techniques.
It is a hands-on workshop using paper-prototyping methods to explore manifesting data as physical objects – why would we do this? what sort of objects? how could it benefit data analysis and computation methods? – and is an opportunity to speculate on how we might discover new ways to generate insights into complex data sets to discern previously undetected patterns and make meaning.
The workshop starts off by engaging the participants in identifying 11 major human senses (touch, taste, smell, sight, hearing, temperature, balance, pain, time, proprioception and interception) and discussing other sensory factors. Then we begin to map how many senses we actually use in everyday activities and tasks. In this way it becomes apparent how reliant we are on multiples senses to interpret our experiences of the world around us. How much then are we missing in trying to analyse data using just our visual and occasionally auditory senses? What patterns might also exist that we are simply not able to perceive because the senses that would detect them are not being activated? The last part of this exercise asks the participants to think about data sources and types, such as different kinds of sensors or data feeds/streams as well as the kinds of data coming through – energy use, health, environmental, sales, traffic, communications etc. We then follow on by mapping how we might interrogate such data using additional senses to sight and sound. What benefits might we get from having new ways to explore big and complex data sets? What could happen when we take digital data out of the machine and into the physical world?
This exercise is followed by a short presentation on Proboscis’ Lifestreams project: how and why we developed the life charms from biosensor data and what implications we believe it has for creating new insights into health and fitness data for wellbeing. Lifestreams provides both an actual context and talking point to discuss the difference that embodying data in the physical world – making it tangible to the senses – could have.
The second workshop exercise involves each participant using a blank StoryCube to imagine a data object of their own. I encourage them to use the 6 sides of the cube to indicate data types or streams that they might be using in their existing work or projects which could be used to generate a hypothetical data object. Then, placing their cube on a worksheet, the participants are prompted to consider how their data objects would connect with different human senses; how the objects might interact or connect with each other; whether they are personal or shared objects; what kind of conditions might people encounter them in. Participants are encouraged to consider what implications may arise from all these too.
Finally we discuss the ideas that have emerged from the exercise and, more generally, the potential for new insights into complex datasets to emerge from creating the possibility for senses other than sight and hearing to be involved in analysis. The point of the co-discovery workshop is to allow participants to come to their own understanding of the potential and what might be possible, not to be didactic. It aims to plant a seed of curiosity by exploring the gaps and absences in our toolkits for creating new kinds of knowledge, hopefully to inspire entirely new ways of expressing data in physical or environmental forms such that we can move beyond the ghetto of the primacy of the screen.
I am available to deliver the workshop to other groups (academic, cultural or corporate). Please contact me for prices and bookings. The workshop lasts about 3 hours, is suitable for a range of abilities and works best with groups of 10 and more. It has been designed to engage scientists at both postgraduate student (Masters/PhD) and professional staff level (lecturer/researcher). It works equally well with designers, artists and others who are also exploring the use of data as a creative material/medium.