Exploring Tensorflow for Poets, or, Building a Pottery Classifier

Exploring Tensorflow for Poets, or, Building a Pottery Classifier

reblogged from our project website at bonetrade.github.io

In our recent paper, ‘Fleshing Out the Bones’ we used the trained ‘Inception 3’ model as a way of determining clusters of images that we then studied for clues and hints: why did the machine cluster them this way? What are the common features?

Unsupervised learning: It’s not unlike reading entrails.

Diagram of the sheep's liver found near Piacenza with Etruscan inscriptions on the bronze sheep's Liver of Piacenza, Wikipedia

An alternate approach is to take an existing model, and add new training data to it. Pete Warden put together a tutorial a few years ago called Tensorflow for Poets that has since been formalized as a Google CodeLabs tutorial. Last night I tried the tutorial out using a corpus of Roman fabrics and wares. The nuts-and-bolts of doing this are over in the tutorials.

The hardest part was getting the training data organized. It needs to be in a folder where each image is in a subfolder where the name of the subfolder is the category, eg:


|
|-training-images
     |
     |-terrasig
     |-african_red_slip
     |-veranice_nera

…etc. Once that was done, it went quite smoothly. What will take some time is figuring out what the different architecture and other flags do. For instance, in the default command suggested by the tutorial, I had to determine that I needed to add the flag on validation size and set it to use the entire training set (as my training set is probably way too small).

python -m scripts.retrain \
  --bottleneck_dir=tf_files/bottlenecks \
  --how_many_training_steps=500 \
  --model_dir=tf_files/models/ \
  --summaries_dir=tf_files/training_summaries/mobilenet_0.50_224 \
  --output_graph=tf_files/retrained_graph.pb \
  --output_labels=tf_files/retrained_labels.txt \
  --architecture mobilenet_0.50_224 \
  --validation_batch_size=-1 \
  --image_dir=tf_files/gallery

“The architecture flag is where we tell the retraining script which version of MobileNet we want to use. The 1.0 corresponds to the width multiplier, and can be 1.0, 0.75, 0.50 or 0.25. The 224 corresponds to image resolution, and can be 224, 192, 160 or 128. For example, to train the smallest version, you’d use –architecture mobilenet_0.25_128.” Harvey, 2017

I’m not sure I understand exactly what this all means yet, practically speaking. Anyway, I now know what I’ll need our MA research assistants to do this fall.

  1. study our first corpus’ clustering results to come up with some training categories
  2. divide that corpus into a training dataset
  3. train a new image classifier
  4. run that classifier on our new corpus (which I’m still collecting)
  5. compare that with the clustering approach without our new classifier.
  6. compare the results of both with the posts’ text

The ambition? To be able to sort at scale the images we find automatically into sensible structure.

 

Featured Image by Adrien Ledoux on Unsplash

Advertisements

MA in History, Public History, Digital Humanities: 2 Positions

MA in History, Public History, Digital Humanities: 2 Positions

Damien Huffer and I are working on a project that elaborates from our ‘Insta-Dead‘ work, which data mined Instagram to explore the trade in human remains. We have an opportunity for two potential MA students to start September 2018 to work with us, at Carleton University.  At the same time, the students would pursue their own research within the ambit of this project, which revolves around the use of various AI technologies – especially, but not limited to, various kinds of neural networks. Ideally, the students’ own research projects would push the research into other domains, for instance, historical photographs; tourist photos; advertising using historical imagery, digital historical consciousness.

Interested candidates are invited to contact Dr. Shawn Graham at shawn dot graham at carleton dot ca to discuss their potential research project, to gauge their potential fit with the funding envelope and other potential supplementary funding sources. Candidates are also invited to review the MA History, MA Public History, with Collaborative Digital Humanities Program requirements –

https://carleton.ca/history/graduate/ma-program/program-requirements/m-a-with-specialization-in-digital-humanities/

https://carleton.ca/history/m-a-in-public-history/

Criteria:
– a good first degree in a relevant subject (history, archaeology, etc)
– existing ability in digital humanities methods or issues is desirable, but not critical. Much more important is an ability to think creatively about the problems or potentials of computational viewpoints.

Activities that the students might be involved in:

+ Writing code to generate datasets
+ Developing various NN
+ Analyzing results
+ Ground-truthing training datasets (making sure that training images are properly classified)
+ Curating and preparing materials for data publication in appropriate venues
+ Research and writing of tutorials
+ Research and writing connected with their own research interests as they intersect with this project
+ Communicating the results of research with relevant publics at conferences and other venues

 

featured image by Green Chamelon on Unsplash

Guest Post: Alexis Mawko, Lessons from Warhol

Guest Post: Alexis Mawko, Lessons from Warhol

HIST3814 Critical Making in Digital History has come to an end. Over the duration of the term, the students first digitized something in the physical world, and then progressively manipulated, translated, and transduced the thing into different forms and mashups. The idea was that we would study the way things break, or fail to work how we think they should, to see if we could understand what the fact of the digital does to our work as historians. The final module involved pushing the digital thing back into the real world. I asked student Alexis Mawko if she would share some of her work. What I like about Alexis’ work is the way she uses the entire process as an opportunity to think about her practice as a historian in its ethical context. Below, we crafted a post out of her various process notes and paradata. The final part is her process notes on how to make a hologram projector

The reason I’m painting this way is that I want to be a machine, and I feel that whatever I do and do machine-like is what I want to do.

‘What is Pop Art? Answers from 8 Painters’, Part 1, G. R. Swenson, in Art News 62, November 1963

I deliberately chose Andy Warhol’s “Brillo Soap Pad Boxes” in the National Art Gallery of Canada for the thing for my assignment for multiple reasons. Warhol’s work forced the viewer to contemplate the relationship between art and everyday commercial objects. By replicating commercial packaging and presenting it as an artwork, the viewers and the curators are also forced to contemplate their own definition of what constitutes art.

Maxwell notes in  Power is in the Process: The ACCORD project that “…in the end we value human creativity above machines” (Maxwell, 2107). To me, this is why Andy Warhol’s work is regarded so highly. While he uses machines, helping hands, and pre-existing designs and images, it is the way that he re-purposes these images and objects that is what makes his work creative, and thus, art. The technology at his disposal allows Warhol to “seek to introduce the value of human creativity” (Maxwell, 2107) into otherwise, banal household objects. In addition, as Maxell notes, “despite his attempts to hide behind the machine, we are always aware of the personality who created the work” (Maxwell, 2107), and it is that creative personality that gives life and meaning to the artwork.

I think the framework of ideas can also be applied to work we have been doing with 3D modeling. We are more removed from the subject and we use technology to create a copy of the artifact at hand. Simply copying an object and processing it through a program to create a 3D model seems uncreative, and the final result would not be classified as art by many. But in essence, we are doing exactly as Warhol did; we are reproducing and repurposing and object for own use, and giving it a new meaning. While Warhol wished for his audience to contemplate his work and their own culture, our 3D models function as an informative tool that allows others to discover the artifact virtually. Furthermore, there is much more involvement in the creation process on our part. I had to edit photos and fiddle with the program to get the best result, and it required a substantial amount of trial and error. If the process was completely devoid of human creativity, the program would do a perfect job generating a 3D model the first time.

While making a 3D model of Warhol’s “Brillo Soap Pad Boxes”, I also had to remove the background of the images and subsequently, the context of the sculpture, that is, the gallery and the setting. In doing so I changed the meaning and the intent that the artist originally had for his work. While a viewer can look at my model and see a 3D image of the sculpture, it may not cause them to understand the meaning behind the sculpture, or contemplate why a consumer product would be displayed in the gallery.

While trying to copy the packaging of the Brillo soap pads, Warhol also re-appropriated the meaning of the Brillo boxes themselves and changed its “aura”- that is, its essential meaning (Bruno Latour and Adam Lowe, 2010). While the boxes started out as packaging designed to promote their product as well as the company’s long history in America, they were seen as something completely different, and more sinister, when viewed in the context of the art gallery. Warhol effectively erased decades of the company’s all American history, and the Brillo Soap Boxes became known throughout the public under a different light.

Finally, the final step that the Brillo Soap Pad Boxes have undergone was its recreation as 3D image, making a facsimile out of a facsimile. When made and viewed as a 3D model, part of the history is lost once again, as the viewer cannot appreciate the context in which his sculpture was meant to be seen in (in a gallery, as mentioned previously). Its meaning becomes an appropriation once again, and the 3D model can only be fully used as a visual supplement, and certainly does not compare to the real thing. Part of the fascination that comes with any artwork or artefact is that that art is has witnessed history, or is representation of the artist himself. Without having that authentic connection and without being able to see the original artefact, part of its meaning, quality, and value is lost.

Module 4 

I had the most difficulty deciding what to do for this final module. I was not quite sure what to create nor did I really know which direction I wanted to take my topic in. The reason for my indecisions really boil down to two main causes of concern.

#1 – I was struggling with how to advance my work in a meaningful way. The point of this module was to bring our digital creations into the physical world. Yet, throughout this course and throughout my own journey with my topic, I have become increasingly conscious of the roles we play as historians and of the potential unexpected consequences we face when manipulating things from the past, whether it be a narrative or a physical object. In Module 2, I focused on the multiple narratives that the Brillo boxes convey, and throughout the Brillo box’s many transformations (from consumer item, to fine art, to a digital 3D model) these narratives have shifted or have been lost all together. Questions of ethics arose as I was aware that each transformation utilized a pre-existing work, which was manipulated for the use and intentions of the creator (myself included). Therefore, I was somewhat hesitant to create a physical copy of a digital copy of a physical copy of an already existing object. (Confused? Me too.) I feared that dragging on this process of copying and reproducing would be redundant, unethical, and meaningless.

#2 – I was also concerned about my practical skills when it came to producing a physical rendering of my object. I feel as though the modules have been getting harder and harder in this regard, and I can admit that technological troubleshooting is not my forte, with my pre-existing technological skills being limited to Microsoft Office and Adobe applications. Thus far, I have been impressed with what I have been able to accomplish, and the discussions we have been having in class about “productive failures” have really helped affirm my less than perfect work and have encouraged me to experiment and move forward. However, with this module I was very unsure of what I wanted to accomplish.

In addressing my first concern regarding the redundancy of repetition, I decided that it was best to just fully acknowledge my role as a historian, as a digital creator, and as an “artefact appropriator” and purposefully bring my digital work from Module 1 back into physical space in the most literal way possible. I therefore decided to create a moving hologram of my 3D model of the “Brillo Soap Pad Boxes” from Module 1. While I was still concerned about how I would actually be able to go about doing this (see Process Notes for this journey, below,) I figured that this option at least allowed me to do some hands on work by creating the actual hologram projector out of plastic. While I am not entirely confident with technology, I am confident in my crafting abilities. Worst case scenario, I would make a hologram out of image rather than a video (thankfully it did not come to this.)

The Brillo boxes have gone through so many changes already, I felt as though making my 3D model into a hologram would intensify the removal of its original context as a consumer product, and its later context as an art piece, making it so steeped with hidden and distorted narratives that its presence as a hologram is a statement itself. The hologram is completely removed from its original surrounding context of a kitchen or an art gallery, and so its purpose and history is completely lost to a typical viewer who may not be informed on this specific topic. Furthermore, as a 3D model, the Brillo Soap Pad Boxes are removed from their setting in a gallery, and as a hologram, the Brillo Soap Pad Boxes are removed from their interactive digital space and physically brought into the world of the viewer. The viewer is then faced with this tiny, broken looking model right in front of them, and they are forced to try to understand what they are looking at and how it can possibly be important.

This got me thinking about a lecture from a few weeks back in which it was said that producing copies and displacing context has the potential to have social and ethical implications. In my case, the ethical implications are very apparent to me. Most of this issue I have already addressed in Module 2, and so at this point, taking this historical object and displacing it further only adds to the already existing ethical implications, yet the extremity may help a confused viewer acknowledge this reality. This idea of ethical implications in regards to holograms has been rather relevant in popular culture as of recent, as there has been a fad of bringing deceased musicians “back to life” through the use of holograms on stage. Discourse surrounds this issue as some people think it is tasteless and improper to seemingly “revive” a deceased musician who cannot consent to the performance. This was discussed fairly recently when a projection of Prince was used at the latest Super Bowl half time performance because he was presented alongside Justin Timberlake, someone that Prince was known to have dislike. Therefore, certain types of holograms do have the potential to cause harm and cross ethical and societal boundries.

As with most forms of reproduction, holograms may present some issues to historians in the future. Failure to represent an object in its proper visual form, or in the proper location or context can have detrimental effects. This has been seen with the reproduction of the Triumphal Arch from Palmyra. As Kamash explains in the article ‘Postcard to Palmyra’: bringing the public into debates over post-conflict reconstruction in the Middle East, “the unfaithful reproduction of an ancient landmark that was not displayed in its home country nor with a proper, informative context proved to be emotionally detrimental to the people of the Middle East AND other viewers who realized the problematic reality of the 3D reconstruction.”

Authenticity in historical artefacts are highly valued because they are tied to the historical narratives it represents. Part of Palmyra’s history is that the ruins were destroyed, and some felt as though that reality was taken away by the attempt at a reproduction. A similar comparison can be drawn to the many reproductions of Brillo boxes. However the scope of those who may have been negatively affected is smaller and perhaps even non-existent today as the Brillo boxes remain as an icon of Andy Warhol’s work, and the Brillo Company itself still remains active and successful today. One could even argue that Warhol’s appropriation had a positive impact on Brillo as his work promoted the company and made them visible in the art and culture world. Either way, it is certain that such a simple object has made a large impact in every form that it has taken.

Process Notes – Building a Hologram Projector

Once I decided on what I actually wanted to produce for this module, I starting making the plastic hologram projector that I would need in order to create a hologram. I figured that making this first was the best course of action because it would allow me to test it out with pre-existing hologram videos on YouTube, and allow me to ensure that it works before I start making my own hologram video. I put my crafting and DIY skills to test and scavenged an old CD case to cut up and use for this project. I followed the instructions set out on this website -> https://makerbus.blog/2018/01/19/building-a-diy-smartphone-hologram-projector-is-it-as-cool-as-the-internet-makes-it-look/ . The tutorial was easy follow and the overall task is quite simple. I opted to use CD case for the plastic instead of the thick, industrial grade plastic that was used in the tutorial because the CD case was easily available to me, and I had the time to spend cutting it.

I first created a paper template of a trapezoid that measured 1cm on the top, 4.5 cm on the sides, and 6 cm along the bottom. I then taped the template onto the CD case plastic and traced around it in pen so that I had an outline to follow when cutting. I laid out the templates so that each sides were touching in order to maximize the efficiency of my cutting.

The plastic was, in no doubt, difficult to cut. I sliced along the lines multiple times with an X-acto knife, applying as much force as I could so that each slice would slowly dig further into the plastic. Once I decided that I had cut into the plastic deep enough, I lined the cutting edge along the side of the table and used pliers to carefully break plastic along the straight edge. I have never cut thick plastic or glass before, but I knew that theoretically, this was the best way to do it. Most cuts and breaks came out fairly well, although I did have some corner piece casualties. Luckily these were the top corners, and it would not affect the overall projection. I then sanded down the edges so that they were not so jagged and sharp, and I carefully tapped the pieces together to form an inverted pyramid. I was especially careful with the tape as I did not want dirty tape obscuring the view of the hologram.

I tested out the hologram projector with some videos on YouTube and it worked perfectly! It was definitely mesmerising to watch. I discovered that the best way to view the hologram is by being in a dark room and turning the phone brightness all the way up.

Creating a Hologram Video

This was the part of the project I was dreading, as I had no idea of how to go about it at first. All I knew was that I somehow needed to get a synchronized, moving video of my 3D model. I had no idea how to render my model into a video at first, and no amount of googling really helped. All of the articles that came up provided instructions on how to make a video into a 3D model…so it was not quite what I wanted. I looked to Sketchfab to see if there were any applications available through there, and there were none that I could find. I figured that my next plan would be to screen record myself manually manipulating my Sketchfab model, and using that video file.

In order to do this, I had to figure out how to screen record (something so simple and yet I had no idea how to do it.) After some googling I found out that a screen recording application comes already installed on Windows 10 devices, and works through Xbox Live. All I had to do to bring it up was hit the windows button + the “G” key at the same time. After hitting one button, I was then set to record. After stopping the recording, the video would save automatically, and was accessed through This PC > Videos > Captures. To get my 3D model video, I went onto SketchFab and made the viewer full screen, and recorded myself manually spinning my model to reveal all sides and angles. While crude and somewhat simplistic, I think it worked quite well and the cursor is only slightly visible but not overly distracting.

The second portion of this endeavour required me to reflect the video four ways and have them all play consecutively. While researching how to do this I discovered this amazing tutorial that teaches you how to do this in POWERPOINT.

Link to Tutorial —> https://nutsandboltsspeedtraining.com/powerpoint-tutorials/hologram-video-powerpoint/

This was fantastic as it allowed me to do everything I needed, in an application that I am already very familiar with, therefore I would not have to download new software and learn a new editing interface.

The tutorial is quite straightforward and easy to follow, and I only had difficulties with the step that teaches you how to sync the videos. After some fiddling around I got that hang of it. I also used a triangle shape as a guide in order to ensure that the videos are positioned the same distance away from the center point. I would align the triangle with the X cross section guide and align the bottom of the videos with the straight edge of the triangle (this can be seen in the images linked in the Object Files).

In order to save the video, I needed to export the file as an MP4 video file. I saved the video in a low quality as it was intended to be viewed on a phone and because the synced the videos did not play smoothly while on a high resolution.

Once finalized, I sent to video to myself so that I could open it on my phone and VOILA! It worked! It was rather tiny (as the boxes could not fit any more into the frame) but it worked. I did notice, however, that it was not as easy to view as the YouTube examples were. This was because the background was grey, and not completely black, because of the default Sketchfab background. I hoped that a grey background would not be a problem but a black background turned out to be necessary for optimal viewing.

I figured out how to change the background in Sketchfab by going into “3D Settings” and changing the background colour to black. After I did this, I recorded a new video and repeated the entire video making process. My final video turned out WAY better and my tiny Brillo Soap Pad Boxes can now be viewed spinning around in virtual space.

~o0o~

Kamash, Zena 2017. ‘Postcard to Palmyra’: bringing the public into debates over post-conflict reconstruction in the Middle East. World Archaeology https://doi.org/10.1080/00438243.2017.1406399

Latour, Bruno, and A. Lowe. “The Migration of the Aura – or How to Explore the Original Through Its Facsimiles”, In. T. Bartscherer and R. Coover (editors) Switching Codes. Thinking Through Digital Technology in the Humanities and the Arts, University of Chicago Press pp. 275-297, 2011.

Maxwell, M. 2017 Power is in the Process: The ACCORD project, Internet Archaeology 44. https://doi.org/10.11141/ia.44.10

small acts of disruption in archaeological publishing

small acts of disruption in archaeological publishing

Last month I presented at the Computer Applications in Archaeology Conference in Tübingen. I was in #s1 (full TAGS archive here) on ‘creative disruption’ in digital archaeology. Below, my slides and the speaking notes from my presentation.

slides: http://shawngraham.github.io/presentations/caa.html#/

1.1. first of all – see Katherine Cook’s Twitter Conference Paper because it is a much better discussion than anything I will do here this morning.

1.2. anything I say must be taken with a grain (or more) of salt. White guy on the internet – can get away with a lot that isn’t possible/permitted to folks who don’t look like me. So folks who look like me have to assume the risk so that others can flourish

1.3. A small experiment in failing gloriously in public – i want to put my publishing projects into a framework of information ethics & sensory archaeology.
– ABM: simple rules, complex results;
– small change: unpredictable results:

– complex systems teach us that small changes can lead to startling emergent effects
– i want to think about some small things we can do that disrupt
– the first small thing is to realize that ethical considerations have to be front and centre with digital archae – see the work of Meghan Dennis!

2.  – Allison Parish, poet & programmer at NYU: “a computer program is a way of projecting power. That’s the point of a computer program, to make a decision and then have it happen millions of times. That’s the real ethical dimension of this for me.

3. decisions taken in a digital medium, given the nature of computation (whose fundamental action is to copy), get multiplied in their effects. Hence, the choices, when there is a choice to be made (as there always is), are a force multiplier for what we think is important.

4. start from first principles:

  • fundamental action of computer: to copy
  • fundamental result of copying: connection
  • fundamental consequence of connection: extended sensorium
  • digital archaeology is an extended kind of digital kinaesthetia, we go intothe flow

5. let’s talk about the things of digital:

• Luciano Floridi: treats everything that exists as informational objects or processes (including bio and other entities, right into databases, agents, etc)
• everything that exists, exists in relation to everything else, with at least some minimum worth. Thus anything that destroys or diminishes data is entropy or morally evil
• ‘Information ethics describes a moral universe [an infosphere] in which not only is no being alone, but every being is indeed morally related to other beings, because in their well-being is connected the welfare of the whole system. [journals] are systems that affect larger systems with their actions, affecting themselves as well, since other systems are procedurally and informationally related to them… Information ethics considers moral actions an information process’. (Sicart discussing Floridi, video game ethics 130). He said ‘agents’; I said ‘journals’
• consider: a paywall is an immoral act because it promotes entropy, it diminishes informational entities
• journals, as we know and love them today fundamentally prevent connection; connection is a kind of sensation (especially digitally)

6. hamilakis 2013 – 415 – argues that aesthetics + politics share same ontological ground: the distribution of the sensible – what is allowed to be sensed + experienced + what is not, what is deemed appropriate or permissible to be sensorially appreciated + embodied and by whom – consensus that is established (imposed or tacitly accepted) sensorial order dissensus, the challenge to that order by pushing new + heterodox sensorial regime

7. to stop entropy, we need to restore sensation, that digital archaeological kinaesthesia of distant knowing

hamilakis talks not of senses but of a sensorial field that returns affectivity to our work – “sensoriality + affectivity also enable + invite a radically different approach to the presentation of the archaeological work, be it in scholarly publications, popular authors, or museum displays, thus creating affective instances + environments for diverse publics” h book 20

this is where my projects come into it. I think. You’ll tell me if I’m wrong, eh? 😉

8. ‘epoiesen’ – made – implies sensuous engagement with the past. ‘epoiesen’ on vases: what games were these people playing/ what did ‘epoiesen’ imply for them? about the human body, about innovations of depiction of human form on a medium that traveled – a ceramic social network

9. the first small act of disruption: a focus on the affectivity of digital or other creative work. a focus on process, and design, and valuing subjectivity

10. another small act: …which also ties to video game ethics: games, as the native art form to the digital age, are only ethical if the choices within are meaningful, consequential… thus publishing in Epoiesen also has to give the author a meaningful choice. here, in terms of licensing. author led. form led (text, photos, art, interactives). yes this is harder for me, but so what?

another small act: futureproofing. responsible to the digital entity, don’t lock in scholarship in proprietary formats. reproducibility, replicability guiding poles

11. collaborative reading – hypothesis

12. another small act: reframing peer review. not about quality assurance. not about guarding the borders. but rather about creating new webs of relationships. new conservations. publishing as a starting point, not a finish line. own names. DOIS to recognize the value of the labour, for to fit in the other games.

13. ODATE – some more small acts. Now, Hamilakis points out H book 9 – archaeology a device of modernity that relies on sense of autonomous and disembodied vision – on other hand, that attitude undermined by intensely physical, embodied interaction with things and environments. This is the same idea that Caraher points to in his ‘slow’ archaeology. digital archaeology is slow. (at least, the way I do it)

14. ODATE is a digital archaeology textbook environment that sits in the same sensuous framework as epoiesen. It comes with its own linux-based computer (dhbox.org). Digital archaeology – to learn it, to challenge it, to dispel the magic of Apple’s training us to expect ‘it just works’ – needs us to open the hood. no disembodied distance from the work of the machine. There is a sense of flow that comes working with data and computation that is every bit as sensuous and embodied as ‘dirt’ archaeology.

15. ODATE: digital archaeology is slow. It is organic. Built on github, language of forks and branches is a biological one. ODATE is always going to be wrong and out of date. That is a strength: with github, replicability and reproducibility and cutting and pasting of the bits that work for you . It will grow, there will be multiple copies. There will never be one canonical version of ODATE. That’s a helluva disruption, right there.

16. conclusion – small acts of disruption in archaeological publishing are actually large acts of disruption in how we think about, with, and through digital archaeology. if we think of archaeological publishing in terms of information ethics and archaeological senses, I think there’s one final small act of disruption that flows from that, and it’s this: we all can do this, already.

Tropy – OCR – Notes Workflow?

Tropy – OCR – Notes Workflow?

Here’s the ideal:

  • I take photos of (printed) documents I want with camera phone
  • photos save to google drive
  • Tropy project reads those photos from Google Drive
  • I use Tesseract to OCR those documents
  • The result is added as a note to each document in Tropy

Ideally, that’d all happen automatically. So far, here’s what I can do

  • take photos with the camera
  • find the photos on the camera, upload them to Google Drive
  • in Tropy, I import the photos from that folder
  • in R Studio, I run a batch OCR script that uses Tesseract
  • I manually add the resulting text into the notes field in Tropy

For reference, here’s my batch ocr script:

library("tesseract")
library("magick")
library("magrittr")

# load 'em up
dest <- "/path/to/images"
myfiles <- list.files(path = dest, pattern = "jpg", full.names = TRUE)

# improve the images
# ocr 'em
# write the output to text file

lapply(myfiles, function(i){
text <- image_read(i) %>%
image_resize("3000x") %>%
image_convert(type = 'Grayscale') %>%
image_trim(fuzz = 40) %>%
image_write(format = 'png', density = '300x300') %>%
tesseract::ocr()

outfile <- paste(i,"-ocr.txt",sep="")
cat(text, file=outfile, sep="\n")

})

Etudier

Etudier

Ed Summers is always up to interesting things. Recently, he cooked up something called étudier

 […] a small Python program that uses Selenium and requests-html to drive a non-headless browser to collect a citation graph around a particular Google Scholar citation or set of search results. The resulting network is written out as a Gephi file and a D3 visualization using networkx.

I had a few issues getting it to upgrade properly (I was there for version 0.0.1!) because I’m frankly a bit messy when it comes to installing and upgrading python things. This led to a lot of frustration, as these things often do, and reminded me forcefully of the advice to be using virtualenv to wall off these different experiments! Below are my notes for getting things running tickety-boo.

pip3 install --upgrade virtualenv

virtualenv edsu-etudier

source edsu-etudier/bin/activate

pip3 install etudier

I already had chromedriver installed using brew install chromedriver so I was ahead of the game there.

Now, let’s find the citation graph for something fun like Hamilakis’ Archaeology and the senses

etudier.py 'https://scholar.google.com/scholar?cites=4956378495941026486&as_sdt=2005&sciodt=0,5&hl=en

ta da! So below, just a quick ‘n’ nasty viz…


feature image ‘Reading Glasses’ Mari Helin-Tuominen, Unsplash
 

Using a Static Site Generator to Make a Nicer Omeka Front Page

Using a Static Site Generator to Make a Nicer Omeka Front Page

 

I like Omeka. But I’m not much good at theme development or customization. John Stewart, on the DH Slack,  showed me some Omeka sites he and his colleagues have been building (like this one and this one) that used static site generators to create lovely front page / splash pages, linking out to the exhibitions and collections.  I was showing these to some of my current students who are working on Omeka projects as part of their MA theses; they liked them, and so we decided to give it a shot.

I initially thought it’d be a simple matter of putting the generated site in the top folder of the domain, and swapping in a new index.html. Turns out, a bit more complicated. John patiently walked me through what he did – thanks John! – and here are my notes for future reference.

We’re using Hugo to generate the static site. Because Jekyll is awful and why would you do that to yourself? We followed the Hugo Quickstart  up to step three. At that point, we cd into our Hugo project’s themes folder, git clone a theme we liked, and copy its config.toml file from its ExampleSite folder into our main project folder. We adjusted the settings there the way we wanted until we got the site looking nice. The hugo command generated the static site into the public folder. Now, this is where things got a little complicated.

When Omeka serves up a site, it generates the look-and-feel from the files in the folder containing the current theme. That’s where our static site has to go. In our case, we were using the default Berlin theme. We’re also using Reclaim Hosting, so navigating around these folders is easy using the FileManager from the cPanel. Go to the relevant theme (for the purposes of this example, the Berlin theme), and you’ll see an index.php file and a css folder. We don’t want our static site to interfere with the Berlin theme inside our omeka site – we’re only messing around with making a nice splash page, remember – so we have to rename our generated css folder in the Hugo site, and then make sure the file paths in our index.html file point correctly. So:

  • once you’ve generated the static site, in the public folder of the Hugo project on your computer:
    • rename the css folder to css2
    • rename index.html to index.php
    • in the index.php file, change the paths to your css folder to css2
    • make sure the filepaths to your css2 folder are pointing to the right place: <link rel="stylesheet" href="http://your-omeka-site.com/themes/berlin/css2/main.css"/>
    • check your file for any other calls to that css2 folder and change them up accordingly
    • zip the contents of the public folder into a zipfile
  • in the theme folder in your omeka installation (here, the Berlin theme),
    • rename index.php to index2.php
  • using the filemanager, upload the zip file into the theme folder
  • extract all (right-click on the file).

And now you have a lovely splash page!

Banner image Andrew Neel, Unsplash

 

Postscript July 5th. For reasons I cannot fathom, my index.php file on my demo site magically reverted to the original php, thus breaking my lovely static front page. This also happened on one of my students’ site. Anyway, check the index.php if for some reason the default omeka front page has made a return on your site.

Tropy to Scrivener Workflow

Tropy to Scrivener Workflow

Scrivener is a great tool for writing, especially if you’re like me and you like to chunk out your thoughts so that you can rearrange them later. Scrivener also has a very nice ‘research’ folder, into which you can copy out your notes a la one-thought-one-card, and then drag them into the actual writing as necessary.

Tropy is a new offering from the Roy Rosenzweig Center for History and New Media at George Mason University. It lets you manage your research photographs, such that you can annotate them, transcribe the text (say you’re in an archive taking pics of historical documents), or otherwise mark them up. I recently thought to myself, facing a stack of short-term loan books I’d accumulated, maybe I can take pictures of the bits here that I’m interested in, and use Tropy to sort them out. (I took the pics with my phone, saving them to my google drive which then synced to this computer where Tropy was waiting).

 

Then I wondered, perhaps I can export the notes such that they update in Scrivener as unique cards? Now, I know that Tropy has a sqlite database in the back end, and presumably I could’ve written some sort of query that’d do all of what I’m about to lay out. But I don’t know how to do that sort of thing. So instead, I’m using jq (see this tutorial) to do the heavy lifting, and a few other commands at the terminal.

1. So make your notes in Tropy. Highlight your images, and select ‘export’. Tropy exports in json-ld which is a bit different than regular ol’ json.

2. I’m on a Mac. Open terminal at the folder where you’re working. We’ll want to remove the top level [ and ], like so:

sed -i.bak 's|\[||' export.jsonld 
sed -i.bak 's|\]||' export.jsonld 

3. Now we’re going to use the jq command (which you can install with homebrew) to get the notes out. This will do the trick:

.["@graph"][].photo[]  | {id: .title, note: .note.html}

So the full jq command in the terminal is:

jq -r '.["@graph"][].photo[] | {id: .title, note: .note.html}' export.jsonld > out.txt

4. Now we’re going to split that out.txt file into separate files. Don’t do it yet, but the command will look like this:

split -p ^{ out.txt new

Let’s make a new folder for the split-up notes that we will then import into Scrivener.

mkdir outputnotes
cd outputnotes

5. Let’s split that out.txt file and have the output written into our current directory:

split -p ^{ ../out.txt

6. Scrivener is probably smart enough to recognize text, even without an extension, but just in case, let’s pretend these things are in markdown. Maybe you actually wrote your notes in markdown in the first place.

for f in *; do mv "$f" "$f.md"; done

7. And so, over in Scrivener, just import this folder!

Or, if you want to use scrivener sync (say you’re collaborating): in scrivener sync settings, make a new folder. Do the jq step, then cd into the new folder (here, ‘scrivtest’). Inside that new folder,

mkdir Notes
cd Notes
split -p ^{ ../../out.txt

You don’t want to be in the Draft folder that scrivener made. Give those files the md extension as before. Now go over to scrivener and hit sync. make sure to tick off the option to sync all other text objets to the project. Ta da! your notes are now in your research folder!

I’m sure this can probably be streamlined, but not bad for an hour’s futzing.

 

3d models from archival film/video footage

3d models from archival film/video footage

Yesterday, I helped Andrew troubleshoot some workflow regarding vr-to-real-world photogrammetry. You should go read his post. As I was doing that, I was thinking that the same flow would work for archival video (which I’ve done with visualSFM, but not Regard3d, so challenge accepted! By the way, the VSFM workflow was Ryan’s regarding models from drones).  So I grabbed some aerial photography of Pompeii from WWII era ish, and gave it a spin. It worked, but it was an ugly ‘beta’-worked, so I left my machine running over the weekend and I’ll know by Monday whether or not the result is any better. I wrote up the workflow, thinking it’d be useful for my class, and deposited with Humanities Commons. I pasted it below, as well. Lemme know if it works for you, or if I’ve missed something.

~o0o~

It is possible to make 3d models from archival film/video footage, although the quality of the resulting model may require a significant amount of sculpting work afterwards to achieve a desireable effect. It depends, really, on why one wants to build a 3d model in the first place. Archaeologists for instance might want to work with a 3d rendering of a building or site now lost.

The workflow
The workflow has a number of steps:

1. obtaining the video (if it is on eg. youtube)
2. slicing the video into still images
3. adding camera metadata to the images
4. computing matched points across the images
5. triangulation from the matched points
6. surface reconstruction

Necessary software
nb these are all open-source or free-to-use programs

1. Youtube-dl https://rg3.github.io/youtube-dl/
2. ffmepg https://www.ffmpeg.org/
3. exiftool https://www.sno.phy.queensu.ca/~phil/exiftool/
4. regard3d http://www.regard3d.org/
5. meshlab (for post-processing) http://www.meshlab.net/

Step One Downloading from Youtube

Archival or interesting footage of all kinds may be found on youtube and other video streaming services. Youtube-dl is a sophisticated program for downloading this footage (and other associated metadata) from youtube and some other sites. Find a video of interest. Note the url. Then:

youtube-dl https://www.youtube.com/watch?v=nSB2VeTeXXg

Try to find video that does not have watermarks (the example above has a watermark and probably is not the best source video one could use). Look for videos that are composed of long cuts, that sweep smoothly around the site/object/target of interest. You may wish to note the timing of interesting shots, as you can download or clip the video to those passages (see the youtube-dl documentation)

Step Two Slicing the Video into Stills

ffmepg is a powerful package for manipulating video and audio. We use it to cut the video into slices. Consult the full documentation to work out how to slice at say every 5 seconds or 10 seconds (whatever is appropriate to your video). Make a new directory in the folder where you’ve downloaded the video with mkdir images. Then the command below slices at every second, numbers the slices and puts them into the frames subdirectory:

ffmpeg -i "downloaded-film.mp4" -r 1 frames\images-%04d.jpeg

Windows users would call ffmpeg with ffmepg.exe (if they haven’t put it into their system’s path variable). Step Three Adding Camera Metadata

We will be using Regard3d to stitch the images together. Regard3d needs to know the camera make, model, focal length (mm), and sensor width (mm). We are going to fudge this information with our best approximation. ‘Sensor width’ is the width of the actual piece of hardware in a digital camera upon which light falls. You’ll have to do some searching to work out the best approximation for this measurement for the likely camera used to make the video you’re interested in.

Find the camera database that Regard3d uses (see the documentation for Regard3d for the location on your system). It is a csv file. Open it with a text editor (eg Sublime Text or Atom. not Excel, because Excel will introduce errors). Add the make, model, and sensor width information following this pattern:

make;model;width-in-mm

Regard3d reads the exif image metadata to work out which camera settings to use. Focal length is read from the exif metadata as well. We assign these like so, from the command line in your frames folder:

exiftool -FocalLength="3.97" *.jpeg
exiftool -Make="CameraMake" *.jpeg
exiftool -Model="CameraModel" *.jpeg

Note that the make and model must absolutely match what you put into the camera database csv file – uppercase, lowercase, etc matters. Also, Windows users might have to rename downloaded exiftool file to exiftool.exe and put it into their path variable (alternatively, rename it and then put it in the frames folder so that when you type the command, your system can find it easily).

Step Four Computing Matches

Open Regard3d and start a new project. Add a photoset by selecting your frames directory. Note that when you used the exiftool, the original images were copied within the folder with a new name. Don’t select those original images. As the images load up, you will see whether or not your metadata is being correctly read. If you get NaN under make, model, focal length, or sensor width, revisit step three again carefully. Click ok to use the images.

Click on compute matches. Slide the keypoint density sliders (two sliders) all the way to ‘ultra’. You can try with just the default values at first, which is faster, but using ‘ultra’ means we get as many data points as possible, which can be necessary given our source images.

This might take some time. When it is finished, proceed through the next steps as Regard3d presents them to you (the options in the bottom left panel of the program are context-specific. If you want to revisit a previous step and try different settings, select the results from that step in the inspector panel top left to redo).

The final procedure in model generation is to compute the surfaces. When you click on the ‘surface’ button (having just completed the ‘densification’ step), make sure to tick off the ‘texture’ radio button. When this step is complete, you can hit the ‘export’ button. The model will be in your project folder – .obj, .stl., and .png. To share the model on something like Sketchfab.com zip these three files into a single zip folder. On sketchfab, you upload the zip folder.

Step Five Clean Up

Double click on the .obj file in your project folder. Meshlab will open and display your model. The exact tools you might wish to use to enhance or clean up your model depends very much on how your model turned out. At the very least, you’ll use the ‘vertice select’ tool (which allows you to draw a box over the offending part) and the ‘vertice delete’ tool. Search the web for help and examples for the effective use of Meshlab.

Regard3d

Regard3d

I’m trying out Regard3d, an open-source photogrammetry tool. A couple of items, memo-to-self style of thing:

    • its database does not have cellphone cameras in it. Had to google around to find the details on my particular phone
    • its database is this: https://github.com/openMVG/CameraSensorSizeDatabase 
    • just had to find where it was on my machine, and then make an entry for my phone. I’m still not sure whether I got the correct ‘width’ dimension – running with this. 
    • nb don’t do this with excel – excel does weird things to csv files, including hidden characters and so on which will cause Regard to not recognize your new database entry. Use Sublime Text or another text editor to make any changes. You can double click on an image in the imageset list inside Regard and add the relevant info one pic at a time, but this didn’t work for me.
    • I took the images with Scann3d, which made a great model out of them. But its pricing model doesn’t let me get the model out. So, found the folder on the phone with the images, uploaded to google drive, then downloaded. (Another nice thing about Scann3d is when you’re taking pictures, it has an on-screen red-dot/green-dot thingy that lets you know when you’re getting good overlap.)
    • Once I had the images on my machine, I needed to add exif metadata re focal length.  Downloaded, installed, exiftool. Command:  exiftool -FocalLength="3.97" *.jpg
    • In Regard3d, loaded the picture set in.
    • The next stages were a bit finicky (tutorial) – just clicking the obvious button would give an error, but if I had one of the image files selected in the dialogue box, all would work.
    • here’s a shot of the process in…erm… process…

  • Console would shout ‘error! error!’ from time to time, yet all continued to work…

I’m pretty sure I saw an ‘export to meshlab’ button go by at some point… but at any rate, at the end of the process I have a model in .ply and .obj!  (ah, found it: it’s one of the options when you’re ready to create the surface). All in all, a nice piece of software.