Stanford NER, extracting & visualizing patterns

This is just a quick note while I’m thinking about this. I say ‘visualizing’ patterns, but there are of course many ways of doing that. Here, I’m just going quick’n’dirty into a network. Say you have the diplomatic correspondence of the Republic of Texas, and you suspect that there might be interesting patterns in the places named over time. You can use the Stanford Named Entity Recognition package to extract locations. Then, using some regular expressions, you can transform that output into a network file. BUT – and this is important – it’s a format that carries some baggage of … Continue reading Stanford NER, extracting & visualizing patterns

The Web of Authors for Wikipedia’s Archaeology Page

I’m playing with a new toy, WikiImporter, which allows me to download the network of authorship on media-wiki powered sites. I fired it up, set it to grab the user-article network and “The Hyperlink Coauthorship network will analyze all the links found in the seed article and create an edge between each user that edited the article found in that link and the article”. Naturally, I pointed it at ‘archaeology’ on Wikipedia.  I’ve posted the resulting two mode network on figshare for all and sundry to analyze. I also asked it to download the article to article links (which is slightly different … Continue reading The Web of Authors for Wikipedia’s Archaeology Page

Beyond the Spaghetti Monster

No, I don’t mean that spaghetti monster. I mean the one that people invoke when they wish to disparage network analysis. That particular spaghetti monster is some variant of a force-directed layout algorithm. Now, these have their place, but they sometimes obscure more than they illuminate. There are alternatives, and Elijah Meeks has been sharing some d3.js code for making interactive ‘arc diagrams’ and ‘adjacency matrices’ that highlight important patterns in network data without the monstrousness. Elijah writes: An arc diagram is another way of visualizing networks that doesn’t use force-directed principles. Instead, it draws the edge from one node … Continue reading Beyond the Spaghetti Monster

Mapping the Web in Real Time

I don’t think I’ve shared my workflow before for mapping the structure of a webcrawl. After listening to Sebastian Heath speak at #dapw it occurred to me that it might be useful for, interalia linked open data type resources. So, here’s what you do (and my example draw’s from this year’s SAA 2014 blogging archaeology session blog-o-sphere): 1. install the http graph generator from the gephi plugin marketplace. 2. download the navicrawler + firefox portable zip file at the top of this page. 3. make sure no other instance of firefox is open. Open firefox portable. DO NOT click the … Continue reading Mapping the Web in Real Time

Gaze & Eonydis for Archaeological Data

I’m experimenting with Clement Levallois‘ data mining tools ‘Gaze‘ and ‘Eonydis‘. I created a table with some mock archaeological data in it: artefact, findspot, and date range for the artefact. More on dates in a moment. Here’s the fake dataset. Firstly, Gaze will take a list of nodes (source, target), and create a network where the source nodes are connected to each other by virtue of sharing a common target. Clement explains: Paul,dog Paul, hamster Paul,cat Gerald,cat Gerald,dog Marie,horse Donald,squirrel Donald,cat … In this case, it is interesting to get a network made of Paul, Gerald, Marie and Donald (sources … Continue reading Gaze & Eonydis for Archaeological Data

Getting Historical Network Data into Gephi

I’m running a workshop next week on getting started with networks & gephi. Below, please find my first pass at a largely self-directed tutorial. This may eventually get incorporated into the Macroscope. Data files for this tutorial may be found here. There’s a pdf/pptx with the images below, too. The data for this exercise comes from Peter Holdsworth’s MA dissertation research, which Peter shared on Figshare here. Peter was interested in the social networks surrounding ideas of commemoration of the centenerary of the War of 1812, in 1912. He studied the membership rolls for women’s service organization in Ontario both … Continue reading Getting Historical Network Data into Gephi

Reanimating Networks with Agent Modeling

I’m presenting next week at the Society for American Archaeology Annual Meeting. I’m giving two papers. One argues for parsimonious models when we do agent based modeling.  The other reverses the flow of archaeological network analysis and instead of finding nets in the archaeology, I use agent based models to generate networks that help me understand the archaeology. (The session is ‘Connected Past’.) Here is the draft of my talk, with all the usual caveats that that entails. Parts of it have been drawn from an unpublished piece that discusses this methodology and the results in much greater detail. It … Continue reading Reanimating Networks with Agent Modeling

Hodder’s ‘Tanglegram’ as Network

I am reading Ian Hodder’s book, ‘Entangled: An Archaeology of the Relationship between Humans and Things’ Hodder writes that the tanglegram cannot be represented as a network, since a network doesn’t consider the nature of the relationships or nodes. This is not in fact the case. Representing these complex relationships as a network is quite possible, and allows the ‘tanglegram’ to actually become a object to query in its own right, rather than a suggestive illustration. I’ve uploaded the network data to Figshare: I used NodeXL to enter the data. If there was a bidirectional tie, I made two … Continue reading Hodder’s ‘Tanglegram’ as Network

Teaching Network Analysis

I had a conversation with Scott Weingart the other day, prompted by this plaintive cry: Brain is broken this AM. Need suggestions for inclass exercises to teach SNA. Can't depend on there being computers: must be analog. Help? — Shawn Graham (@electricarchaeo) October 15, 2012 Backstory: I’m teaching a class where we are looking at maps and networks and archaeological data, as ways of understanding how cities and countryside blur into one another in the ancient world. Last week, we played iterated Prisoner’s Dilemma’s with playing cards (thanks to this site by Alannah Morrison) as part of a discussion about … Continue reading Teaching Network Analysis

Coins from Sirmium in the PAS database

Interested in networks, and looking for an exemplar that I could do in my class, I turned to the Portable Antiquities Scheme database, and extracted coins known to have originated from the mint at Sirmium (modern Sremska Mitrovica, Serbia). You can find the list here. I downloaded the data as a CSV. Looking at it, it seemed to me that a multimodal graph of coin to findspot, to material, to date, and to ruling house might be useful (and of course could be transformed into single mode graphs as necessary).  So I made a list, where the 21 coins were … Continue reading Coins from Sirmium in the PAS database