I’ve sussed the Scrivener syncing issue by moving the process of converting out of the syncing folder (remember, not the actual project folder, but the ‘sync to external folder’). I then have created four automator applications to push my stuff to github in lovely markdown. Another thing I’ve learned today: when writing in Scrivener, just keep your formatting simple. Don’t use markdown syntax within Scrivener or your stuff on github will end up looking like this \##second-heading. I mean, it’s still legible, but not as legible as we’d like.

So – I have four robots. I write in Scrivener, keep my notes, close the session, whereupon it syncs rtf to the ‘external folder’ (in this case, my dropbox folder for this purpose; again, not the actual scrivener project folder).

1. I hit robot 1 on my desktop. Right now, this is called ‘abm-project-move-to-conversion-folder’. When I have a new project, I just open this application in Automator, and change the source directory to that project’s Scrivener external syncing folder. It grabs everything out of that folder, and copies it into a ‘conversion-folder’ that lives on my machine.
2. I hit robot 2, ‘convert-rtf-to-md’, which opens ‘conversion-folder’ and turns everything it finds into markdown. The conversion scripts live in the ‘conversion-folder’; the things to be converted live in a subfolder, conversion-folder/draft
3. I hit robot 3, ‘push-converted-files-to-github-repo’. This grabs just the markdown files, and copies them into my local github repository for the project. When I have a new project, I’d have to change this application to point to the new folder. This also overwrites anything with the same file name.
4. I hit robot 4, ‘clean-conversion-folder’ which moves everything (rtfs, mds,) to the trash. This is necessary because if not, then I can end up with duplicates of files I haven’t actually modified getting through my pipeline onto my github page. (If you look at some of my experiments on github, you’ll see the same card a number of times with 1…2…3…4 versions).

Maybe it’s possible to create a meta-automator that strings those four robots into 1. I’ll try that someday.
[pause]
Ok, so of course, I tried stringing them just now. And it didn’t work. So I put that automator into the trash –
[pause]
and now my original four robots give me errors, ‘the application …. can’t be opened. -1712’. I found the solution here (basically, go to spotlight, type in activity, then locate the application on the list and quit it).

Here are my automators:

Automator….

I think I love you.

 

I like Scrivener. I *really* like being able to have my research and my writing in the same place, and most of all, I like being able to re-arrange the cards until I start to see the ideas fall into place.

I’m a bit of a visual learner, I suppose. (Which makes it ironic that I so rarely provide screenshots here. But I digress). What I’ve been looking for is a way to share my research, my lab notes, my digital ephemera in a single notebook. Lots of examples are out there, but another criterion is that I need to be able to set something up that my students might possibly be able to replicate.

So my requirements:

1. Visually see my notes, their layout, their possible logical connections. The ability to rearrange my notes provides the framework for my later written outputs.

2. Get my notes (but not all of the other bits and pieces) onto the web in such a way that each note becomes a citable object, with revision history freely available.

3. Ideally, that could then feed into some sort of shiny interface for others’ browsing – something like Jeckyll, I guess – but not really a big deal at the moment.

So #1 is taken care of with Scrivener. Number 2? I’m thinking Github. Number 3? We’ll worry about that some other day. There are Scrivener project templates that can be dropped into a Github repository (see previous post). You would create a folder/repo on your computer, drop the template into that, and write away to your hearts content, committing and syncing at the end of the day. This is what you’d get. All those slashes and curly brackets tell Scrivener what’s going on, but it’s not all that nice to read. (After all, that solution is about revision history, not open notebooks).

Now, it is possible to manually compile your whole document, or bits at a time, into markdown files and to commit/sync those. That’s nice, but time consuming. What I think I need is some way to turn Scrivener’s rtf’s into nice markdown. I found this, a collection of scripts by Sparkygetsthegirl as part of a Scrivener to Android tablet and back writing flow. Check it out! Here’s how it works. NB, this is all Mac based, today.

1. Make a new Scrivener project.

2. Sync it to dropbox. (which is nice: backups, portability via Dropbox, sharing via Github! see below)

3. drop the 4 scripts into the synced folder. Open a terminal window there. We’ll come back to that.

4. open Automator. What we’re going to do is create an application that will open the ‘drafts’ folder in the synced project, grab everything, then filter for just the markdown files we made, then move them over to our github repo, overwriting any pre-existing files there. Here’s a screenshot of what that application looks like in the Automator editing screen:

You drag the drafts folder into the ‘Get specified finder items’ box, get the folder contents, filter for files with file extension .md, and then copy to your github repo. Tick off the overwrite checkbox.

Back in scrivener, you start to write.

Write write write.

Here’s a screenshot of how I’m setting up a new project.

In this screenshot, I’ve already moved my notecards from ‘research’ into ‘draft’. In a final compile, I’d edit things heavily, add bits and pieces to connect the thoughts, shuffle them around, etc. But right now, you can see one main card that identifies the project and the pertinent information surrounding it (like for instance, when I’m supposed to have this thing done). I can compile just that card into multimarkdown, and save it directly to the github repository as readme.md.

Now the day is done, I’m finished writing/researching/playing. I sync the project one last time. Then, in the terminal window, I can type

./rtf2md Draft/*.rtf

for everything in the draft folder, and

./rtf2md Notes/*.rtf

for everything in the notes folder. Mirabile dictu, the resulting md files will have the title of the notecard as their file name!

Here, I’ve used some basic citation info as the name for each card; a better idea might be to include tags in there too. Hey, this is all still improv theatre.

Now, when I created that application using automator, I saved it to my desktop. I double-click on it, and it strains out the md files and moves them over to my github repository. I then commit & sync, and I now have an open lab notebook on the web. Now, there are still some glitches; my markdown syntax that I wrote in, in Scrivener, isn’t being recognized on github because I think Scrivener is adding backslashes here and there, which are working like escape characters?

Anyway, this seems a promising start. When I do further analysis in R, or build a model in Netlogo, I can record my observations this way, create an R notebook with knitr or a netlogo applet, and push these into subfolders in this repo. Thus the whole thing will stick together.

I think this works.

~o~
Update Sept 18. I’ve discovered that I might have messed something up with my syncing. It could be I’ve just done something foolish locally or it might be something with my workflow. I’m investigating, but the upshot is, I got an error when I synced and a new folder called ‘Trashed Files’, and well, I think I’m close to my ideal setup, but there’s still something wonky. Stay tuned.

Update Sept 19 Don’t write in Scrivener using markdown syntax! I had a ‘doh’ moment. Write in Scrivener using bold, italics, bullets, etc to mark up your text. Then, when the script converts to markdown, it’ll format it correctly – which means that github will render it more or less correctly, making your notes a whole lot easier to read. Click on ‘raw’ on this page to see what I mean!