by Lev Manovich
What is “big social data”? The following description from the 2011 grant competition
organized by a number of research agencies (including National Endowment for
Humanities and National Science Foundation) in USA, Canada, UK, and Netherlands
provides an excellent description:
“The idea behind the Digging into Data Challenge is to address how « big data » changes the research landscape for the humanities and social sciences. Now that we have massive databases of materials used by scholars in the humanities and social sciences — ranging from digitized books, newspapers, and music to transactional data like web searches, sensor data or cell phone records — what new, computationally-based research methods might we apply? As the world becomes increasingly digital, new techniques will be needed to search, analyze, and understand these everyday materials.” www.diggingintodata.org (accessed March 31, 2011).
In this article I want to address some of the theoretical and practical issues raised by emerging “big data”-driven social science and humanities. My observations are based on my own experience over last three years with big data projects carried out in my lab at UCSD and Calit2 (softwarestudies.com). The issues which we will discuss include the
differences between “deep data” about a few and “surface data” about the many; getting access to transactional data; and the new “data analysis divide” between data experts and the rest of us