historical maps into Unity3d

This should work.

Say there’s a historical map that you want to digitize.  It may or may not have contour lines on it, but there is some indication of the topography (hatching or shading or what not). Say you wanted to digitize it such that a person could explore its conception of geography from a first person perspective.

Here’s a workflow for making that happen.

Some time ago, the folks at the NYPL put together a tutorial explaining how to turn such a map into a minecraft world. So let’s do the first part of their tutorial. In essence, what we do is take the georectified map (which you could georectify using something like the Harvard Map Warper), load that into QGIS, add elevation points, generate a surface from that elevation, turn it into grayscale, export that image, convert to raw format, import into Unity3d.

Easy peasy.

For the first part, we follow the NYPL:

Requirements

QGIS 2.2.0 ( http://qgis.org )

  • Activate Contour plugin
  • Activate GRASS plugin if not already activated

A map image to work from

  • We used a geo-rectified TIFF exported from this map but any high rez scan of a map with elevation data and features will suffice.

Process:

Layer > Add Raster Layer > [select rectified tiff]

  • Repeat for each tiff to be analyzed

Layer > New > New Shapefile Layer

  • Type: Point
  • New Attribute: add ‘elevation’ type whole number
  • remove id

Contour (plugin)

  • Vector Layer: choose points layer just created
  • Data field: elevation
  • Number: at least 20 (maybe.. number of distinct elevations + 2)
  • Layer name: default is fine

Export and import contours as vector layer:

  • right click save (e.g. port-washington-contours.shp)
  • May report error like “Only 19 of 20 features written.” Doesn’t seem to matter much

Layer > Add Vector Layer > [add .shp layer just exported]

Edit Current Grass Region (to reduce rendering time)

  • clip to minimal lat longs

Open Grass Tools

  • Modules List: Select “v.in.ogr.qgis”
  • Select recently added contours layer
  • Run, View output, and close

Open Grass Tools

  • Modules List: Select “v.to.rast.attr”
  • Name of input vector map: (layer just generated)
  • Attribute field: elevation
  • Run, View output, and close

Open Grass Tools

  • Modules List: Select “r.surf.contour”
  • Name of existing raster map containing colors: (layer just generated)
  • Run (will take a while), View output, and close

Hide points and contours (and anything else above bw elevation image) Project > Save as Image

You may want to create a cropped version of the result to remove un-analyzed/messy edges

As I noted a while ago, there are some “hidden, tacit bits [concerning] installing the Contour plugin, and working with GRASS tools (especially the bit about ‘editing the current grass region’, which always is fiddly, I find).”  Unhelpfully, I didn’t write down what these were.

Anyway, now that you have a grayscale image, open it in Gimp (or Photoshop; if you do have photoshop go watch this video and you’re done.).

For those of us without photoshop, this next bit comes from the addendum to a previous post of mine:L

    1. open the grayscale image in Gimp.
    2. resized the image as power of 2 + 1 (*shrug* everything indicates this is what you do, with unity); in this case I chose 1025.
    3. save as file type RAW. IMPORTANT: in the dialogue that opens, set ‘RGB save type to ‘planar’.
    4. Change the file extension from .data to .raw in mac Finder or windows Explorer.

Now you can import this historical elevation map in Unity. In Unity, add a gameobject -> 3d object -> terrain to the project. In the inspector window, there’s a cogwheel. Click this; it opens the settings. One of the options will be ‘import raw’. Click this.

Select your .raw grayscale image.

  1. On the import dialogue, change it to 8-bit image rather than 16-bit.
  2. Change the width, height, x and z to all be 1025. Changed the y to be 75 (yours will be different; look at the range in your original map of highest and lowest point, and input that. For reference, please also see this post which saved me: http://newton64.github.io/blog/2013-07-24-gimp-unity-terrain.html

Ta da – a white glacial landscape with your elevation data.

Screen Shot 2015-06-09 at 12.14.30 PMNow the fun stuff can happen. But – before someone can ‘walk’ around your landscape, you have to add controls to your project. So, in Unity 3d, go to:

Assets – Import package – characters.

Once that’s all done, you’ll drag-and-drop a ‘FPSController’ into your project. You’ll find it as below:

Screen Shot 2015-06-09 at 7.26.52 PM

Click and grab that blue box and move it up into your project (just drop it in the main window). Make sure that the control is above (and also, not intersecting any part of) your landscape, or when you go to play, you’ll either be stuck or indeed falling to the centre of the earth. We don’t want that. Also, delete the ‘camera’ from the hierarchy; the fpscontroller has its own camera. My interface looks like this:

Screen Shot 2015-06-09 at 7.30.47 PM

You do the grass and trees etc from the terrain inspector, as in the window there on the right. I’ll play some more with that aspect, report back soonish. Notice the column drum in the right foreground, and the tombstone in the back? Those were made with 3d photogrammetry; both are hosted on Sketchfab, as it happens. Anyway, in Meshlab I converted from .obj to .dae, after having reduced the polygons with quadratic edge decimation, to make them a bit simpler. You can add such models to your landscape by dropping the folder into the ‘assets’ folder of your Unity project (via the mac Finder or windows explorer).  Then, as you did with the fpscontroller block, you drag them into your scene and reposition them as you want.

Here’s my version, pushed to webGL

Enjoy!

(by the way, it occurs to me that you could use that workflow to visualize damned near anything that can be mapped, not just geography. Convert the output of a topic model into a grayscale elevation map; take a network and add elevation points to match betweeness metrics…)

Historical Maps, Topography, Into Minecraft: QGIS

Building your Minecraft Topography

(An earlier version of this uses Microdem, which is just a huge page in the butt. I re-wrote this using Qgis, for my hist3812a students. If you’d like to see what some of them accomplished, head over to the github repo where there’s ‘Slave of Portus’, ‘Vimy Ridge’, and ‘Crafting the Canal’)

If you are trying to recreate a world as recorded in a historical map, then modern topography isn’t what you want. Instead, you need to create a blank, flat world in Worldpainter, and then import your historical map as an overlay. In worldpainter, File >> New World. In the dialogue box, uncheck ‘circular world’. Tick of ‘flat’ under topography. Then, on the main icon ribbon, select the ‘picture frame’ icon (‘image overlay’). In the dialogue box, tick ‘image overlay’. Select your file. You might have to fiddle with the scale and the x, y offset to get it exactly positioned where you want. Watch the video mentioned below to see all this in action. Then you can paint the terrain type (including water), raise, lower the terrain accordingly, put down blocks to indicate buildings… Worldpainter is pretty powerful.

If you already have elevation data as greyscale .bmp or .tiff

  • Watch the video about using Worldpainter.
  • Skip ahead to where he imports the topographic data and then the historical map imagery and shows you how to paint this against your topography.
  • You should also google for Worldpainter tutorials.

If you have an ARCGIS shapefile

This was cooked up for me by Joel Rivard, one of our GIS & Map specialists in the Library. He writes,

  • Using QGIS: In the menu, go to Layer > Add Vector Layer. Find the point shapefile that has the elevation information.
  • Ensure that you select point in the file type.
  • In the menu, go to Raster > Interpolation.
  • Select “Field 3″ (this corresponds to the z or elevation field) for Interpolation attribute and click on “Add”.
  • Feel free to keep the rest as default and save the output file as an Image (bmp, jpg or any other raster)

If you need to get topographic data

In some situations, modern topography is just what you need.

  • Grab Shuttle Radar Topography Mission data for the area you are interested in (it downloads as a tiff.) To help you orient yourself, click off ‘toggle cities’ at the bottom of that page. You then click on the tile that contains the region your are interested in. This is a large piece of geography; we’ll trim in a moment.
  • Open QGIS
  • Go to Layer >> Add Raster Layer. Navigate to the location where your srtm download is located. You’re looking for the .tiff file. Select that file.

Add Raster Layer

  • You now have a grayscale image in your QGIS workspace, which might look like this

Straights of Hercules, Spain, Morocco

  • Now you need to crop this image to just the part that you are interested in. On the main menu ribbon, select Raster >> Extraction >> Clipper

Select Clipper Tool

  • In the dialogue box that opens, make sure that ‘Clipping Mode’ is set to ‘Extent’. With this dialogue box open, you can click and drag on the image to highlight the area you wish to crop to. The extent coordinates will fill in automatically.

  • Hit ‘Select…’ beside ‘Output File’. Give your new cropped image a useful name. Hit ‘Save’.

  • Nothing much will appear to happen – but on the main QGIS window, under ‘layers’ a new layer will be listed.

Imgur

  • UNCHECK the original layer (which will have a name like srtm_36_05). Suddenly, only your cropped image is left on the screen. Use the magnifying glass with the plus sign (in the icons at the top of the window) to zoom so that your cropped image fills as much of the screen as possible.
  • Go to Project >> Save as image. Give it a useful name, and make sure to set ‘files of type’ to .bmp. You can now import the .bmp file to your Worldpainter file.

Importing your grayscale DEM to a Minecraft World

Video tutorial again – never mind the bit where he talks about getting the topographic data at the beginning

At this point, the easiest thing to do is to use WorldPainter. It’s free, but you can donate to its developers to help them maintain and update it. Now, the video shown above shows how to load your DEM image into WorldPainter. It parses the black-to-white pixel values and turns them into elevations. You have the option of setting where ‘sea level’ is on your map (so elevations below that point are covered with water). There are many, many options here; play with it! Adam Clarke, who made the video, suggests scaling up your image to 900%, but I’ve found that that makes absolutely monstrous worlds. You’ll have to play around to see what makes most sense for you, but with real-world data of any area larger than a few kilometres on a side, I think 100 to 200% is fine.

So: in Worldpainter – File >> Import >> Height map. In the dialogue box that opens, select your bmp file. You’ll probably need to reduce the vertical scale a bit. Play around.

Now, the crucial bit for us: you can import an image into WorldPainter to use as an overlay to guide the placement of blocks, terrain, buildings, whatever. So, again, rather than me simply regurgitating what Adam narrates, go watch the video. Save as a .world file for editing; export to Minecraft when you’re ready (be warned: big maps can take a very long time to render. That’s another reason why I don’t scale up the way Adam suggests).

Save your .world file regularly. EXPORT your minecraft world to the saves folder (the link shows where this can be found.

Go play.

Wait, what about the historical maps again?

The video covers it much better than I could here. Watch it, but skip ahead to the map overlay section. See the bit at the top of this post.

Ps. Here’s Vimy Ridge, site of a rather important battle in WW1 fought by the Canadian Army, imported into Minecraft this way:
Vimy Ridge in Minecraft