Comments on: Political Tweets

By: Kathleen Hulser

Kathleen Hulser — Sun, 08 Nov 2009 16:22:08 +0000

Should we pity the researcher of the future browsing these political tweets, or feel relieved that at least some written communications will still be written and saved unlike the vast majority of email info? Abundance and relevance, the promise and the pitfall of the future.

By: Arian D. Ravanbakhsh

Arian D. Ravanbakhsh — Sat, 07 Nov 2009 13:37:23 +0000

Yes, I’m the NARA archivist that can’t sleep because of Twitter and frankly, almost all “web 2.0′ sites. And the standard disclaimer applies – what follows is my thinking and not NARA policy.

To answer Julianna’s point, I’ll say that she’s right about the need to archive Twitter and the potential interest in Twitter down the road. From a broader, government-wide perspective, the same thinking applies as well. For instance, most public statements (press releases, annual reports) from Federal agencies are generally appraised as permanent records. It’s then an easy leap to translate that to the Twitter feed created by an agency – such as NASA or the CDC. It’s a little awkward when the agencies don’t own the servers that they are posting to, but as everyone has mentioned, tools exist that allow the capture of tweets. Personally, I’ve used Tweetake to get a CSV backup and it worked pretty well.

Twitter is actually pretty straightforward on both the appraisal and the technology side. Where it gets more difficult is where the content is not as easy to appraise, let alone capture – think Facebook. That’s where archivists can argue about appraisal theory and still not agree. Some agencies use FB as a place to aggregate their press releases and such. You can make the argument that if that’s the case, then as long as the original press releases are captured in other places, then the content on FB is not a record. But what do you do with comments that exist solely on FB at that point?

I’m glad that students like Julianna are thinking about archival preservation on web 2.0 platforms. That bodes well for the profession moving forward. I also feel that traditional archival appraisal theory has to evolve as well. We’re seeing the first steps of that evolution. Keep up the good work. Now, I’ll go back to worrying about how to capture content from Second Life.

By: Amanda French

Amanda French — Thu, 05 Nov 2009 22:29:05 +0000

I’ve used Twapperkeeper before; it’ll output RSS and tab-delimited text, but one major limitation is that it only archives things that are marked with #hashtags (Twitter’s organic form of metadata).

By: Jennifer Waxman

Jennifer Waxman — Thu, 05 Nov 2009 21:56:43 +0000

As I recall, the LOC used the web crawler Heretix to archive the Sotomayor tweets. They explain the entire process on their digital projects page. Two other methods of archiving tweets involve other open source applications. One is called TheArchivist. You can export into Excel and keep your searches ongoing as long as you open the search to gather every day. It will get tweets up to two weeks back or up to 1500, which ever comes first. Obvious problems are if you loose your connection or your computer crashes, you could obviously not have a “complete” record of the tweets you are following. I tested it out during the Iran election riots to see how it works. It works, but the results should be further analyzed to determine how well it holds up as a born-digital record.

The other option is the Twapperkeeper, but I have less experience with that and do not believe it’s as easily exported. Many folks dealing with digital preservation and web archiving are researching ways to archive web 2.0 content. It’s a very new field and for obvious reasons, will be very important down the road.

By: Amanda French

Amanda French — Thu, 05 Nov 2009 21:40:34 +0000

Thanks for being our first blogger, Julianna! I will say that I know that some archivists are kind of tearing their hair out over archiving Twitter — I met someone at the recent MARAC who works for NARA who said that this kind of thing keeps him up at night. The Library of Congress did archive some tweets related to the nomination of Sonia Sotomayor, as I understand, but I don’t know of anyone who’s doing individual twitter feeds.

I do know that right now if you want to archive a Twitter feed, you have to set it up beforehand — one way it can be done is by subscribing to the feed with an RSS reader such as Google Reader and then outputting the feed as plain text.