I’ve been looking for an open notebook solution for some time. Tonight, I think I’ve hit a combination of tools that are sufficiently powerful and straightforward enough that I can integrate them into my undergraduate teaching. But first:

Caleb McDaniel’s eloquent argument for why one would want to do it is here: http://wcm1.web.rice.edu/open-notebook-history.html

Carl Boettiger is another inspiration: http://www.carlboettiger.info/lab-notebook.html

Mark Madsen’s explanation of how his open notebook works – and the logic he organizes it by – is similarly an inspiration: http://notebook.madsenlab.org/labnotebook.html

and Ben Marwick’s work is pathbreaking (especially within archaeology): http://rstudio-pubs-static.s3.amazonaws.com/14247_4703e332c133404b9765f61082dd54cc.html

Right. So what is this solution? Zettlekasten + Pykwiki. In an earlier post I got the one-card-per-note ‘zettlekasten’ system that Dan Sheffler uses working on my machine (by the way, you will learn a *lot* if you spend some time reading Dan’s thoughts on the art of note taking). In essence, Dan’s system is just a simple plugin for Sublime Text that designates a particular folder a ‘wiki’. When I type [[ Sublime text shows me a quick list of every markdown file in that folder which I can select to make a link. The link then shows up in my file as [[something like this]] and I can ctrl click on that link to jump to the other file. If there is no file when I hit [[ I can click to create a new file with the text between the [[ and ]] as the file name. It’s a bit like Notational Velocity in that regard (and of course, you could use Notational Velocity if you wanted, though – I’d have to check – I don’t think you can save your notes as separate .md files update: yes, yes you can. In ‘preferences, notes, storage’ there are two lists of file types. Hit the + button under each one in turn, adding ‘md’. Then, highlight the other options and hit the – button. Folks just getting started – start with nvAlt. You can also crosslink as described above in nvAlt using the [[ and typing the first letters of the card you wish to crosslink to).

So, with a bit of forethought and sensible naming conventions, I can quickly build up quite a network of cards. Dan also has a script for exporting annotations made on pdfs from Skim as markdown files (my post on this), and the resulting md file can be chopped up quickly or integrated into the existing network of cards).

This is good. What is even better is being able to keep this structure, searchable, online in a way that could be forked. This is where Pykwiki comes into the picture. Pykwiki is a static site generator, run from the command line, that will take a folder of .md files and generate a site from them, which you can then put on your own server space (see a list of features here). Now, I could just put my Zettlekasten in a github repo in the first place, and then push them online. But the files would lose that network of connections – remember, Sublime is interpreting my [[ ]] as an internal link. Pykwiki *also* understands those [[ ]] as internal links! (caveat: don’t have any spaces in file names).

So here’s the set up. I install Pykwiki. I designate Pykwiki’s ‘source’ folder as my Zettlekasten folder for Sublime text. I do my readings, I make my notes, I generate my web of notecards. I make sure to have a ‘data block’ for each note, which is [[ and ]] again, above and below the metadata, with the body of the card below – at a minimum, it looks like this:

[[
title: Space Syntax in Pompeii - shortcomings
]]

## Laurence, Space & Society in Roman Pompeii

+ On page 194....

…but can also include tags and other kinds of meta data that’d be useful to have. Also, a notecard can be designated ‘private’ so it doesn’t show up in search, but is still findable if you know the direct url. (And I’m also using another script from Dan Sheffler which associates BibDesk cite keys with custom URLs to open up the pdfs I was reading in the first place, so my online notes will open my pdfs on this machine or any other one where I have that script installed.).  With my session ended, I make sure to save all my cards, close Sublime, and go over to the terminal, and ‘pykwiki cache’ – et voila! The site is generated. It’s now in the ‘docroot’ folder. This folder is pushed to a github pages branch, and I’ve got myself a searchable open notebook wiki (with rss feed! So recipes from IFTTT.com can be used to further mash things up). (Set up on Mac is easy; Windows not so easy, but here are instructions in my open notebook.)

Sweet!

(PS Thought occurs – since I sometimes also do work in R, I can probably get that whole Rmd scene working with this as well.)

(PPS At this point, you might reasonably expect to find a link to my open online notebook. Erm. Well, I only got all the bugs out of the flow this evening. Over the coming days, I’ll start trying to put some of my existing notes – or maybe, start a new notebook – up.)

update oct 9 Getting pykwiki set up on a mac is easy; getting it set up on windows is more difficult, although this could just be because I’ve missed something fundamental. I don’t use any sort of bash on my windows machine, which might be the source of some of my issues. Anyway, I got it working on windows, and the instructions are in *my open notebook[!]*. http://shawngraham.github.io/installing-pykwiki-on-windows.html.

In terms of making your notebook live on the interwebs, say on a github pages site- the default pykwiki settings (see the config.yaml file) work out-of-the-box if and only if you move the static files generated (in the docroot folder) to the root of the domain (or subdomain). Ie, http://shawngraham.github.io works out of the box, but shawngraham.github.io/example-folder does not. This location: example-pykwiki-notebook.example.com would also work (ie a subdomain). To get folder paths to work, you have to update the config.yaml’s ‘web_prefix’ settings- see http://shawngraham.github.io/weprefix.html .

ppps the files I have up at the moment come from an earlier experiment with mdwiki as an open notebook solution. They are currently horrible because they’ve undergone some nasty translations back and forth across scrivener on mac and windows, an android device, and a few other tortured byways. I’m in the process of cleaning them up, crosslinking them, and tagging them, which has the virtue of reminding me what I’ve written and what I was thinking about a year ago. Incidentally, I set up the ‘source’ folder to be its own repository on my github, so I have two repos – the one that serves the generated site, the other that has the clean .md files. These latter I could copy into a scrivener project, and use them to begin writing. Win!

another update – I was reading this: http://www.chs-fellows.org/2015/08/03/contextualizing-digital-data-as-scholarship-in-eastern-mediterranean-archaeology/ by Eric Kansa, which led me to wonder this:

https://twitter.com/electricarchaeo/status/652550926717353992

which led to an awesome conversation, and also this exciting piece of news:

https://twitter.com/erochest/status/652551892690714625

…so you should trace those conversations back. Also, I made a note in my open notebook about it all:

http://shawngraham.github.io/thoughts%20on%20a%20coin.html

(open notebook image: https://www.flickr.com/photos/barnimages/18368762634/sizes/c/ public domain)