Text Analysis of the Grand Jury Documents

a topic in the grand jury documents, #ferguson
a topic in the grand jury documents, #ferguson

I watched Twitter and the CBC while the prosecutor was reading his statement. I watched the live feeds from Ferguson, and other cities around the US. Back in August, when this all first began, I was glued to my computer, several feeds going at once.

A spectator.

Yesterday, Mitch Fraas put the grand jury documents (transcripts of the statements, the proceedings) into Voyant Tools:

These ultimately came from here: http://apps.stlpublicradio.org/ferguson-project/evidence.html

So today, I began, in a small way, to try to make sense of it all, the only way that I can. Text analysis.

Here’s the Voyant Tools corpus

https://twitter.com/electricarchaeo/status/537632916357251072

https://twitter.com/electricarchaeo/status/537636614026317824

https://twitter.com/electricarchaeo/status/537637104801824768

https://twitter.com/electricarchaeo/status/537637583187345409

Not having read the full corpus closely (this is, of course, a *distant* tool), it certainly looks as if the focus was on working out what Brown was doing, rather than Wilson…

I started topic modeling, using R & MALLET.

https://twitter.com/electricarchaeo/status/537703186371719169

and I put everything up on github

https://twitter.com/electricarchaeo/status/537705271318290432

but then I felt that I could improve the analysis; I created one concatenated file, then broke it into 1000 line chunks. The latest inputs, outputs, and scripts, are all on my github page.

https://twitter.com/electricarchaeo/status/537778448237076480

The most haunting…

https://twitter.com/electricarchaeo/status/537777278315663360

And all 100 topics…
https://twitter.com/electricarchaeo/status/537778166631530498

None of this counts as analysis. But – by putting it altogether, my hope is that more people will grab the text files, grab the R script, explore the Voyant corpus, and really put this all under the microscope. I was tremendously effected by Bethany’s latest blog post, ‘All at once‘, which discusses her own reaction to recent news in both Ferguson and UVa, and elsewhere. It was this bit at the end that really resonated:

[…]we need analytical and interpretive platforms, too, that help us embrace our own subjective positioning in the systems in which we labor–which means, inevitably, to embrace our own complicity and culpability in them. And we need these, at the same time, to help us see beyond: to see patterns and trends, to read close and distantly all at once, to know how to act and what to do next. We need platforms that help us understand the workings of the cogs, of which we are one.

So here’s my small contribution. Maybe this can be a platform for someone to do a deeper analysis, to get started with text analysis, to read distantly and closely, to see beyond, and to understand what happened during the Grand Jury.

Advertisements