Value, Vision, and Big Data
Editor • ONLINE
Everywhere I look, the phrase “Big Data” pops up. It seems a very sudden ascent in popularity. I confirmed it’s not just a subjective feeling on my part by doing a rudimentary bibliometric analysis using ABI/INFORM and Computer Database. In 2005, scant attention was paid to Big Data, with only 59 mentions in the business database and 10 in the computer file. The action picked up in 2011, with 697 in ABI/INFORM and 720 in Computer Database. The numbers have doubled for 2012, and we’re only partially into the year.
I’m very tempted to say that librarians understood Big Data before Big Data was cool—and before it gained such popularity. I do believe that information professionals were comfortable with large data sets before anyone else, but Big Data? Did we really get that? I’m not convinced. Certainly, the premium content databases libraries subscribe to contain massive amounts of data. Is it petabytes? Zettabytes? Does it matter if we’re not adequately using new discovery tools?
Laura Gordon-Murnane makes the point in her Big Data article in this issue that Big Data has the potential to change what we do. It presents opportunities for information professionals to become data librarians and curators—people who can help maximize the understanding of data and fully exploit its value. Preserving data to perpetuate its use and reuse is another skill set possessed by librarians.
I think the point of Big Data is not that it’s big. I’d focus instead on the data. It’s unstructured; it’s unlike what we cut our teeth on with subscription databases; and it cries out for new methods of analysis. Big Data brings opportunities, but new uses of data lead to new insights.
Marcy Phelps brings insights to both Big Data and a paucity of data through visualizations, while David Stuart warns that leaving the development of new technologies in the hands of large companies could violate basic librarian principles regarding privacy and unfettered and unfiltered access to information. Both Stephen Arnold and Mary Ellen Bates mention I.B.M.’s Watson, the machine personification of Big Data: Steve marvels at Big Blue’s acceptance of open source software, and Mary Ellen speculates on how this fits with empathic search.
My concerns regarding Big Data are somewhat different. I agree that information professionals have a unique set of skills that manipulators of big data should rely upon and cherish. However, our value as analysts will be valued only if we have the vision to see Big Data as not confined to information retrieval or library collections—it’s customer preferences, tweets, student course choices, traffic patterns, crime data, astronomy, and more. It’s predictions rather than retrieval, trend spotting across huge quantities of data rather than reading a few pertinent articles, and engaging with new forms of search technology that will create opportunities for information professionals. Information professionals must seize these opportunities and convince those outside the profession that we bring significant value to the Big Data world.