digital history, making

Procedural History

I see this is my third post with this title. Ah well. Just playing with a script I found here

shawngraham$ python2 historygen.py
In the beginning there was “Baubrugrend Free State”, “Dominion of Clioriwen”
In that era, the people could bear it no longer, and so these ones rebelled from ‘Baubrugrend Free State’ ==> ‘Province of Vrevrela’
In that era, the people could bear it no longer, and so these ones rebelled from ‘Province of Vrevrela’ ==> ‘Free People’s Republic of Craepai’
It is a terrible thing when brothers fight. Thus ‘Free People’s Republic of Craepai’ became “Eiwerela”, “Broteuvallia”
It is a terrible thing when brothers fight. Thus ‘Dominion of Clioriwen’ became “Duchy of Corica”, “Orican Republic”
The thirst for new lands, new glory, and the desire to distract the people, led to new conquests ‘Duchy of Corica’ conquered ‘Eiwerela’
The thirst for new lands, new glory, and the desire to distract the people, led to new conquests ‘Duchy of Corica’ conquered ‘Broteuvallia’
The thirst for new lands, new glory, and the desire to distract the people, led to new conquests ‘Duchy of Corica’ conquered ‘Orican Republic’
In that era, the people could bear it no longer, and so these ones rebelled from ‘Duchy of Corica’ ==> ‘United States of Heukan’
In that era, the people could bear it no longer, and so these ones rebelled from ‘United States of Heukan’ ==> ‘Kingdom of Amoth’
END “Kingdom of Amoth”

The script can also make a nice diagram; now to get it to write the history AND the diagram at the same time.

The directionality of the arrows is a bit confusing. You almost have to read it backwards. However, since it is just a .dot file, I think I can probably load it into something like yEd and make a prettier timeline.

update I’ve added Tracery to the script, made the output a bit more lyrical:

> shawngraham$ python2 historygen.py

Gather by, young ones, and let me tell you of our nations and peoples.

In the beginning there was “Duchy of Corica”

These people shared a single peninsula, shielded from the rest of the world by tall mountains.

Flooding ruined the crops; the famine weakened them all and so, ‘Duchy of Corica’ dissolved in fragments, eventually becoming “Province of Eabloris” and “Voches” and “Uamafai “

A few years later, the strength of the people could bear it no longer, and they rose up in violent revolution. The old ‘Province of Eabloris’ was no more; a new dawn broke on ‘Heawoth’.

As it came to pass, the Queen gave up power and fled into exile. The old ‘Heawoth’ was no more; a new dawn broke on ‘Iroa’

Flooding ruined the crops; the famine weakened them all and so,“Uamafai ” and “Voches” became ‘Eiwerela’.

As it came to pass, the Satrap gave up power and fled into exile. The old ‘Eiwerela’ was no more; a new dawn broke on ‘Oyune’

Low cunning and high treachery divided them and so, ‘Oyune’ dissolved in fragments, eventually becoming “Broteuvallia” and “Islands of Hekla” and “Kingdom of Abroth”.

Low cunning and high treachery divided them and so, ‘Islands of Hekla’ dissolved in fragments, eventually becoming “Satrapy of Yaislaxuin” and “Dominion of Clioriwen”.

The clouds grew dark, and hunger stalked the land, so sickness weakened them all and so, “Dominion of Clioriwen” and “Satrapy of Yaislaxuin” became ‘Kingdom of Amoth’.

The thirst for new lands, new glory, and the desire to distract the people, led to new conquests. ‘Broteuvallia’ conquered ‘Kingdom of Amoth’

A few years later, the Queen gave up power and fled into exile. The old ‘Iroa’ was no more; a new dawn broke on ‘Province of Vrevrela’

Standing proud upon the ruins there are only now “Broteuvallia”and “Kingdom of Abroth”and “Province of Vrevrela”.

(feature image: Chester Alvarez, Unsplash

Advertisements
Standard
Uncategorized

Procedural History

Procjam 2017 – ‘make something that makes something!’ is on right now.  My interest in procedural generation at the moment concerns the way we think about something when we’re making something else that makes that something. (Still with me?)

I’ve dabbled in sound, and in text, and in bots; now I’m thinking about procedural history. Now, in a way, that was what I was doing when I got into this whole DH scene way back when – agent based modeling. But there’s something about the way something like Dwarf Fortress writes history – and people in turn flesh those histories out (this was the subject of an undergraduate honours thesis done for me by a student, who, to my frustration, has never posted the work online).

So I’m coming back to it. This is also partly because I’m interested in how ideas about how cities work are codified in city sim games that then get draped in the trappings of Antiquity (am writing a piece on this at the moment). Today I came across a very cool project from the procjam community, by David Masad (who I note is using ABM for his dissertation work), called ‘WorldBuilding‘. It takes the algorithm for fantasy maps that you may have seen at play in that twitter account, ‘uncharted atlas’, generates a world by simulating landscape and erosion, introduces an ABM of nomads who settle down into cities, who interacted via trade routes, and who pay tribute or go to war with one another.

All within one python notebook.

Here is the dusty plateau, fringed by a verdent coast, where one or two valleys give access to the interior. Nomads arrive, and in the course of time, settlements and routes emerge:

And we can begin to model their interactions.  Masad is using Axelrod’s Tribute Model. He then dips into the logs and is able to generate the annals of this world:

“From 0 to 49, Ceotpe saw slow growth. In this period it received tributes from Oqlou.
From 49 to 50, Ceotpe saw slow decline. In 50 it, and joined its allies in one battle.
From 50 to 87, Ceotpe saw slow growth. In this period it received tributes from Oqlou.
From 87 to 88, Ceotpe saw slow decline. In 88 it, and joined its allies in one battle.
From 88 to 90, Ceotpe saw slow decline. In this period it, and joined its allies in one battle.
From 90 to 92, Ceotpe saw rapid decline. In this period it fought a war against Oqlou.
From 92 to 99, Ceotpe saw slow growth.”

The code also produces a line chart of a city’s fortunes – for instance, the city of Tigei had a much different history:

The logs spell out what was happening:

” 43, ‘Receive tribute’, ‘Oqlou’
50, ‘Joined war against’, ‘Itykca’
62, ‘Receive tribute’, ‘Oqlou’
73, ‘Receive tribute’, ‘Oqlou’
88, ‘Joined war against’, ‘Itzyos’
90, ‘Joined war against’, ‘Itzyos’
92, ‘Led war against’, ‘Oqlou'”

I could imagine, say in a senior undergrad seminar where I had the time and commitment from students, to use this code as the kernal for a deep exploration of how history intersects with games and simulation. How do you get from an annal to a history? The students would work at the creative tension between what a game shows and what a game merely suggests and what the player brings to that gap.

Something like that. It’s dark, it’s november, the time changed, I’m tired, not exactly coherent. But there’s something extremely cool here (not least the usage of jupyter notebook to illuminate the code). My gut, which I consult on such things, thinks this stuff is important.

featured image Peter Lewicki, unspash

Standard
Uncategorized

Old Bones Daily – HeritageJam 2017 Entry

Update: We won!

I’m pleased to take part in the 2017 HeritageJam; Kate and I present Old Bones Daily:

  • Visit online at http://shawngraham.github.io/hj2017
  • Make your browser full-screen for best effect
  • Fully responsive so can be read equally well on mobile
  • Reload the page for the latest headlines and stories and photos

Paradata

The paradata for the project is printed as the fifth column on the newspaper’s front page.

This ‘newspaper’ that you are looking at pays hommage to that older newspaper culture of reprinting, while at the same time commenting, on modern ‘news media’ by procedurally generating texts from the ‘bones’ of generative grammar. Each time this page is reloaded, the news is generated anew from a Tracery grammar. This grammar is derived from the study of 19th century newspapers in Western Quebec (The Equity) and northern New York State. It selects passages from these papers where the word ‘bone’ is present or implied, and recombines them in sometimes surprising or revealing ways. There are stories of injury, and columns with helpful advice. Sometimes a humorous anecdote is recounted; sometimes popular accounts of the latest academic research.

And sometimes, out and out fraud.

Some of the passages are presented verbatim, using the language and reflecting the mores of the age (with no generative grammar intervention) thus demonstrating another perspective on ‘bones’ and whose are accorded human dignity. The passages were collected in the first place by ingesting OCR’d papers into AntConc, and generating a concordance and keywords-in-context file.

Tracery was designed and built by Kate Compton @galaxykate. This newspaper layout was designed and shared by user silkine on codepen.io. Images are drawn from the British Library’s Flickr stream, where they have been tagged with the word ‘bone’ – thus, another algorithmic expression of the bones underlying the web.

Source Files

Please see the source repository at http://github.com/shawngraham/hj2017

To replicate this newspaper for your own amusement, consult the file, newsgrammar.js and insert new values in the keys. To alter the layout and placement of these elements on the newspaper, create new var and associated div in the js\app.js file. Place the div in the appropriate location in the main index.html file. Extra css or html for a particular chunk of text should be wrapped inside the values in the newsgrammar.js file. For a tutorial on how Tracery functions, see Shawn Graham’s tutorial at The Programming Historian. Tracery can power webpages, games, and twitterbots. What would be the effect of ‘Old Bones Daily’ if it were translated into the new news medium of Twitter?

Sources for Newspaper articles:

The Shawville Equity (consult this digital finding aid project by Carleton undergraduate student Jeff Blackadar)

Syracuse NY Daily Journal via ‘Old Fulton NY Post Cards’ collection of digitized newspapers at http://www.fultonhistory.com/

Licensing

The original CSS for the newspaper layout is MIT Licensed by user Silkine on codepen.io

The Tracery generative grammar is released by Kate Compton under Apache License Version 2.0, January 2004

Images obtained via the British Library’s Flickr stream are all public domain works

We assert that our use of the original source newspapers is fair use

We release the code in the source repository into the wild, such parts of it that are uniquely ours, under CC BY.

Standard
Uncategorized

A Twitter Hiatus

https://twitter.com/electricarchaeo/status/918867063577501696

For some time now, Twitter just hasn’t been fun. Don’t get me wrong: if you want to know what’s going on in digital archaeology (or whatever field) and you want to know what your peers are up to, it’s hard to beat.

But it’s been making me sick. It’s been making me anxious. It’s been eating far too much of my time. The world’s shitty enough as it is, without giving the shittiness a direct pipeline into my brain.

There’s no need to iterate all the bad ways Twitter serves to make the world a worse place, including its use of addictive design principles. It’s baked in, as they say: the trolls, the nazis, and Trump will never be booted off, because they generate clicks, they generate attention. Women and people of colour on the other hand…

So I’m taking a break. I’m trying. I’ve installed a site blocker on all my devices. In recognition that I’ve been such a heavy twitter user for so long, and that’s the principle vector for how some people collaborate with me, I have things set up so that I can check my DMs periodically. I did retweet some things related to singer-poet Gord Downie’s death, which as a Canadian of a certain age, well, you just have to do.

Other than that, I’ve been off Twitter for a week now. And dammit if I don’t feel more productive already…

If you need me, email me.

Feature image Vicko Mozara, Unsplash 

 

Standard
archaeology, environments, teaching, Uncategorized

Call for Collaborators: The Open Digital Archaeology Textbook Environment (ODATE)

The Open Digital Archaeology Textbook Environment is a collaborative writing project led by myself, Neha Gupta, Michael Carter, and Beth Compton. (See earlier posts on this project here).  We recognize that this is a pretty big topic to tackle. We would like to invite friends and allies to become co-authors with us. Contact us by Jan 31st; see below.

Here is the current live draft of the textbook. It is, like all live-written openly accessible texts, a thing in the process of becoming, replete with warts, errors, clunky phrasing, and odd memos-to-self. I’m always quietly terrified to share work in progress, but I firmly believe in both the pedagogical and collegial value of such endeavours. While our progress has been a bit slower than one might’ve liked, here is where we currently stand:

  1. We’ve got the framework set up to allow open review and collaboration via the Hypothes.is web annotation framework and the use of Github and gh-pages to serve up the book
  2. The book is written in the bookdown framework with R Markdown and so can have actionable code within it, should the need arise
  3. This also has the happy effect of making collaboration open and transparent (although not necessarily easy)
  4. The DHBox computational environment has been set up and is running on Carleton’s servers. It’s currently behind a firewall, but that’ll be changing at some point during this term (you can road-test things on DHBox)
  5. We are customizing it to add QGIS and VSFM and some other bits and bobs that’d be useful for archaeologists. Suggestions welcome
  6. We ran a test of the DHBox this past summer with 60 students. My gut feeling is that not only did this make teaching easier and keep all the students on the same page, but the students also came away with a better ability to roll with whatever their own computers threw at them.
  7. Of six projected chapters, chapter one is in pretty good – though rough – shape

So, while the majority of this book is being written by Graham, Gupta, Carter and Compton, we know that we are leaving a great deal of material un-discussed. We would be delighted to consider additions to ODATE, if you have particular expertise that you would like to share. As you can see, many sections in this work have yet to be written, and so we would be happy to consider contributions aimed there as well. Keep in mind that we are writing for an introductory audience (who may or may not have foundational digital literacy skills) and that we are writing for a linux-based environment. Whether you are an academic, a professional archaeologist, a graduate student, or a friend of archaeology more generally, we’d be delighted to hear from you.

Please write to Shawn at shawn dot graham at carleton dot ca to discuss your idea and how it might fit into the overall arc of ODATE by January 31st 2018. The primary authors will discuss whether or not to invite a full draft. A full draft will need to be submitted by March 15th 2018. We will then offer feedback. The piece will go up on this draft site by the end of the month, whereupon it will enjoy the same open review as the other parts. Accepted contributors will be listed as full authors, eg ‘Graham, Gupta, Carter, Compton, YOUR NAME, 2018 The Open Digital Archaeology Textbook Environment, eCampusOntario…..

For help on how to fork, edit, make pull requests and so on, please see this repo

 

Featured Image: “My Life Through a Lens”, bamagal, Unsplash

Standard
archaeology, history, making, publishing

Building Epoiesen

The curtain goes up, the first pawn moves, the first shot is fired*—but that’s not the start. The play, the game, the war is just a little window on a ribbon of events that may extend back thousands of years. The point is, there’s always something before. It’s always a case of Now Read On.

Much human ingenuity has gone into finding the ultimate Before.

The current state of knowledge can be summarized thus:

In the beginning, there was nothing, which exploded.

* Probably at the first pawn.

  • Terry Pratchett, Lords and Ladies 

Epoiesen is now alive, a journal for creative engagement in history and archaeology. But when did Epoiesen begin? What was its genesis?

As I look through my notebooks and emails and miscellaneous files, I can’t find the *exact* beginning (I know that I’ve been interested in new publishing models for a while though). I find in my inbox an email setting up a meeting with George Duimovich and Pat Moore from our Library to talk about Open Journal Systems in March of 2015. I find scribbles of ideas in notebooks going back to about 2014 (not coincidentally, shortly after my tenure and promotion portfolio was shoehorned from its born-digital format into dead pdfs). In October 2015, I find a google doc that I shared with some folks for an idea of something called ‘Paradata: A Journal of Digital Scholarship in Archaeology and Ancient History’. The influence of HeritageJam I think is clear too 😉 I wrote,

“I see this idea as being parallel to things like http://openarchaeologydata.metajnl.com/ which publishes ‘data papers’. Paradata would publish the scholarship that goes into making something of that kind of info, while the dead tree versions can be where people duke it out over the interpretations. Moreover, since link rot and website death is a real issue, Paradata would hit a real niche where projects could continue ever after”

But that idea seems to have run out of steam.  I’m not entirely sure why. I find one note that suggests we felt that our idea perhaps was too close to what DHCommons Journal had in mind.

My notes go silent for a while. Then I find scribbles in my notebooks again from around the time of my participation in MSUDAI, the Digital Archaeology Institute at MSU, concomitant with the creation of @tinyarchae my Tracery-powered dysfunctional excavation bot. That was August 2016. Then, sometime in September of last year, I find a website I built:

O what could’ve been, eh? Here, I’m clearly going for a bit of whimsy; not so much just paradata for conventional digital projects, but maybe something more. The core idea still seems to be there – a place to validate digital things.  I rather like that template, and I need to remind myself what I was using to build it. Structurally, there’s a debt here to open research notebooks done in Jekyll, so that’s probably it.  I did show this ‘Miscellaney’ to people, but there were some very strong reactions to the name, to the whimsy, as it were – see below. (I still like ‘Haptic Visions’ as a category though).

The actual email that led to Epoiesen seeing the light of day comes from October 16 2016:

Hi Pat,

As I was saying to George – and I think you and I have talked about this too on occasion – I’ve been interested to explore creating an open access journal for digital archaeology. I’ve seen the open journals platform, and while it is very cool, it’s not quite what I’m thinking of. I’m interested in something a bit more idiosyncratic that would be based on simple text files in the markdown format, and building the site/journal from that with a static site generator.

The idea is to create what amounts to a kind of literary journal, but for creative engagement with the past. I would solicit everything from twitter bots (I’ve created one that tweets out what amounts to a procedurally-generated soap opera, scenes from an excavation) to music, to art, to creative writing, to data viz… I would solicit reviews, but these would also be published alongside the work under the reviewers’ name. The Hypothesis web annotation architecture would also be built in  […] In a way, it would be a place to publish the ‘paradata’ of digital making in archaeology … Does this sound feasible? Is it something we could do? Maybe I could drop by sometime to chat.

Pat said ‘Yes’. Simple word, ‘yes’. Strong word, ‘yes’. Librarians are powerful.

From that initial meeting, many more meetings took place. Research. Emails. Phone calls. I’ll try to summarize… My first port of call was of course those folks who’ve done this kind of thing before. Martin Paul Eve published a series of posts on his blog that offered his advice on starting an open access journal, and I can’t recommend them enough. Indeed, if you’re one of the people who received an email from me about joining the editorial board, you’ll recognize that I adhered rather closely to Eve’s model.

I was still going with the ‘Smith’s’ name until about November of last year, when I find an email I wrote,

I have, on the advice of several people whose situation is far more precarious than my own, gone for a bit of a name change to signal a bit less whimsy….They rightly pointed out to me that as junior folks the perception of their work is everything, and my whimsical ’Smiths’ name would undermine rather than help them…

One of the earliest folks on board was the wonderful Sara Perry.  I find we exchanged several yonks-worth of emails, throwing ideas around about who to contact, who might be persuaded to submit, and so on. The wonderful folks of the editorial board as a group kept me grounded, found potential contributors, suggested Trello as a way of keeping track of who was doing what, and basically helped keep things on track when my enthusiasm threatened to derail things.

While all of this was going on, I continued to play with the design and platform. I eventually settled on Hexo as a static site generator. I’d been using Jekyll with my open research notebook, but Jekyll frankly is just not something I can work with.

Now Hexo is not without its idiosyncracies. I learned how it builds the site out of little snippets of ‘ejs’ code. I learned how to embed (that is, into what ‘partial’ to paste) Hypothes.is . I figured out where to place the bits and bobs of Tipuesearch (an open source jquery search engine plugin) into the site. (It generates a full json representation of all the site content, so not only making it searchable, but other folks can use it for remixing, data viz, whatever). You wouldn’t believe how hard it was to work out how to make the list-of-articles page, list alphabetically, rather than by date. There was also a battle where the hexo deploy command for pushing everything to my test site ingested accidentally a bunch of stuff I didn’t want – super huge image files – and so I had to wade deep into the waters of git to fix (and I thank the DHAnswers channel in DH Slack for the help!).  Turns out, if you’re using Hexo and Github, don’t fiddle with anything via the website. Getting DOIs embedded into the page metadata, that was also difficult.

Here’s what the YAML metadata for an article looks like:

---
title: Destory History
date: 2017-09-01 20:01:04
tags: interactive fiction
cover_index: /imgs/coyne/quinten-de-graaf-258711.jpg
cover_detail: /imgs/coyne/11146352055_64c730a741_o.jpg
author: "Coyne, Lucas"
doi: "10.22215/epoiesen/2017.4"
---

The partials grab the ‘author’ for alphabetizing posts, and the doi for embedding into the metadata:

<% if (page.doi){ %>
        <meta name="dc:identifier" content="<%= page.doi %>" />
    <% } %>

That might not mean very much to you, or look very impressive, but darnit,  it took several hours of reading stackoverflow and fiddling with things, generating the site over and over again, so I’m pasting it here for posterity…. anyway, I now have a default template for creating articles, that has reminders within it of the kinds of information that I need to include and where they go.

One year later, Epoiesen exists in the world! I announced it with a tweet…

https://twitter.com/electricarchaeo/status/913041896238436352

… we will be doing a formal ‘Ta Da!’ during open access week in October. So many people come together at just the right time to make something in this world. Serendipity, and someone says ‘Yes’, and suddenly, there is something that wasn’t before. What else is tenure for, if it isn’t to make space for someone else to do something new? I hope you’ll consider Epoiesen for your own experiments and creative engagements.

~o0o~

I’m grateful to everyone who has sent me a note or tweeted regarding the start of Epoiesen. I look forward to seeing where this adventure will lead! Thank you, all. I’m also grateful to Neville Morley, who writes about Epoiesen’s situation in the broader publishing landscape in ‘Changing the Rules

Moving forward, I’ve very excited to work with Bill Caraher and The Digital Press at the University of North Dakota to publish the Epoiesen Annual, where all of the articles from a given year are gathered together and given (printed) flesh.

Stay tuned! Make wonderful things!

Standard
teaching

Letter to a young scholar

I sometimes receive notes from undergrads or other folks wondering what advice I can give about studying to become X… I thought I’d share the response I wrote this morning.

Hi ____

Thank you for your note, and your query about how I got here and various options for your own path. I’ll tell you first about my own journey. Don’t let that part put you off, but I want you to have your eyes open as you consider your options.

My own personal journey is perhaps not a template to follow: I went to the UK for grad school in Roman archaeology. At the end of that process, I was teaching random courses at universities across the south east of england, piecing together enough money to keep me going, living out of a rucksack. I eventually got tired of that and came back to Canada where I was, for all intents and purposes, unemployable in Canadian archaeology. I started my own businesses, and also supply taught at a local high school, to make ends meet (see this: https://medium.com/@electricarchaeo/on-teaching-high-school-109cb75caedc ). Eventually I got a position working online for a for-profit “university” in the US, which gave me a bit of stability. Eventually, I saw the job advert for a position in ‘digital humanities’ at Carleton, and here I am.

So my journey involved transforming myself from frankly a second rate Roman epigraphist into a digital humanities scholar and digital archaeologist. I benefited from being in the right place at the right time, having made a bit of a name for myself by blogging my continuing research throughout that period. There was a lot of luck involved.

Between December 2002, when I received my PhD, and July 2010, when I started at Carleton, I had precisely 2 interviews for full-time academic postings.

Now, the keys to getting the job at Carleton were that when I returned to Canada, I had to work extremely hard to make connections with people in the community I wanted to be a part of. Conferences, open research online. Contract archaeology wanted nothing to do with me because I had not gained enough experience of field archaeology in the UK to be employable in Canada – AND Canadian archaeology uses different approaches than european stuff.

*my advice, for what it’s worth*

  • I’d have still gone to the UK for grad school, but I would not necessarily jump into doing a PhD. Few places in this world are better for archaeology, ancient civ, etc. An MA opens opportunities; a PhD can be perceived as narrowing your range of options – you have to work hard to convince people of the truth of the phd, that it makes you better in the long run for a wide variety of things.
  • I knew I didn’t want to go to a Canadian school, because I wanted to jump right into my interests. A UK school allows that; Canadian schools demand a whole bunch of coursework first.
  • Follow the money: go where they really want you. If a school offers some sort of scholarship, I’d take it. My 1 year of MA in the UK doubled my entire debt to that point.
  • Do an MA that fills you with joy – it’s one of the few times in this life where you can. An MA of any stripe is all to the good, so don’t fall into the instrumentalist trap of picking something that you think someone ‘else’ (however construed) would approve.
  • A classical MA, of whatever stripe, can be a very good foundation for a wide variety of paths in this life. Don’t worry necessarily about the job at the end of it. Classical folks in my experience tend to be some of the most creative and lateral thinking people I’ve ever met.
  • Be aware that such things can take a toll on your mental health. Make plans to keep your support networks, your friendships, intact
  • I’d have focussed on getting more fieldwork. That said, archaeology suffers from gendered labour issues such that it is largely men in positions of power. So if you plan on trying for an archaeological career in fieldwork, know that this is an issue.
  • Classics departments are greying, but they are not necessarily hiring to replace retirement.
  • Work constantly on your digital literacy: skills, trends, research methods, questions, theories
  • Develop a scholarly online presence
  • Lurk on twitter, follow scholars whose work fills you with wonder, or whom you admire. Follow a couple you loathe, for a contrary view.

You might also wish to frame your interests a bit more broadly, and consider in what other contexts you can engage with Greek and Roman civ – museums, digital work, community, public, game studies, and so on.

Best wishes,

Shawn

(cover image, Daria Nepriakhinia, Unsplash)

Standard
Uncategorized

Data as a Kandinksy Painting

I just found this package for R, ‘Kandinsky‘. You can read the logic of what it does here.

I’m totally into representing data as art, so I thought I would feed all 900+ annotations my ‘Crafting Digital History’ class is making across the web through it

  • Grab all the annotations using Lincoln’s ‘Hypothesisr‘ package.
  • Turn that into tidy data:
word_counts <- documents %>%
  group_by(user) %>%
  unnest_tokens(word, text) %>% 
  count(user, word, sort = TRUE) %>%
  ungroup()
  • feed word_counts into kandinsky

et volia:

 

Now, let’s visualize the stopwords. I also add some custom stopwords to that list (things like ‘digital’, ‘historian’ etc, given nature of the course). Ecco:

There is something extraordinarily satisfying about those two images. The first captures the entire universe of possible responses that my students are making. In the second, that purple circle seems to my mind to correspond with the normal stopwords and the squiggles my additions. Let us now subtract the second from the first:

Interesting, this visualization of what remains after the stopwords are applied…

I can also do some other fun things with my annotations, such as term frequency – inverse distribution frequency to find out what words tend to characterize which students’ annotations. As a Kandinsky painting:

Let’s paint our feelings – here’s the sentiment of the annotations (‘affin’):

And here’s the same data again, but sorted from most positive to most negative:

Finally, let’s finish off with a topic model and then the top terms from the topic model:

Data is beautiful.

What does it mean? Well, that might take another post or two. Maybe the meaning would emerge if I also sonified, or 3d printed, this data. If we use the full sensorium…

Standard
Uncategorized

Uh oh

I’ve taught ‘Crafting Digital History’ twice before. Once as a face to face course complete with lectures and in-class exercises, and once as a fully online course. The workbook now approaches 200 pages when it is printed out. One takeway from the 2016 edition was that I didn’t want to be writing tutorials and supporting students across multiple operating systems.

Especially Windows. Windows drives me up the freakin’ wall.

Because I also like to learn, and I’m trying to push for reproducibility as a goal in digital history (of methods at least, and re-visiting of conclusions) and in digital archaeology, I had it in mind for some time that some sort of virtual machine would be great. Everybody would be on the same platform. I would only have to write one set of materials. But experiments with virtual machines kept throwing up the same issues of getting the damned machine installed and configured correctly across multiple operating systems. I especially loathe those back-to-school specials with 2 gb that so many of my students seem to have (if you only do a bit of wordprocessing and facebook, good enough I suppose).

Enter DHBox.

I love DHBox. I love the concept. I love the philosophy of openness baked in. I decided ‘go big or stay home’ and so I rewrote the 2017 version of the course to use DHBox nearly exclusively. And up until about, oh, 11.30 last night, things were going great.

A troubling error message, but not the end of the world. We had already increased the amount of memory allocated to our DHBox twice already (we have it installed on top of an openstack.org stack). Earlier, in the run up to the course, we tried to estimate how much memory the students would need. I wanted the students to work with real digitized materials that hitherto had not attracted any attention – the Shawville Equity’s print run from 1883-2010. I figured I could teach them how to use wget to download this stuff, and then in the next module I’d teach them various ways of looking at it, exploring it, extracting interesting stuff from it. Earlier, I’d also taught them how to use Twarc to download materials from Twitter, suggesting they use the ‘canada150′ hashtag (Non-Canadians: it’s 150 years from Confederation, whence sprungeth modern-ish Canada).

Oops.

Being only a few weeks from the official day of celebrations (July 1) meant that there were, oh I don’t know, hundreds of thousands of tweets with that tag available via Twarc. Multiply by # of students.

Number of editions of the Equity available for download: 1595. Each one between 8 and 20 high-rez pages. Even though I asked the students to only download a few years’ worth, multiply by malformed Wget and/or processes left running…. (I had shown them and walked them through how to identify and kill running processes when necessary, but alas…)

And so I sent a call out to Andrew who has been supporting this class above and beyond the call of duty. He’s on vacation. But he tried to help me out regardless, and set things in motion to increase our memory allocation. Unfortunately, we’d clogged the pipes so badly that this process has itself gone sideways in ways that I am unable to explain (server-side stuff ain’t my bag, as Austin Powers might say).

And so we are currently DHBox-free. While this has caused me a mild heart-attack, it’s not really as bad as it might first seem. I still have all of my materials written from last year where I was supporting individual operating systems, so I just dusted that off (thank you, O Github repository) and gave it to the students who needed it.

The only thing that is seriously hurt at this point is my pride, and the loss of some downloaded data. The final projects – where I imagined them all collaborating on different aspects of that particular dataset – will need to be rejigged a bit, but it’s all going to be ok.

It’ll be ok.

Right?

….Right?

//

Featured Image by Simson Petrol on Unsplash

Standard