Monday, October 2, 2017

Structure from Motion using video from my phone camera

Introduction. In my last post, I showed how I had extracted frames from satellite-derived video to build a 3D model of a mine from space. In this post, I show a similar workflow for phone camera: take a video of an object (some times a lot easier than numerous frames although lower quality) and make a 3D model using Agisoft Photoscan.

Video to images. (this part is repeated from the last post). The main generic challenge for SfM from video is to extract the video frames and prepare them for the SfM. The SfM part is no different from what my group has been doing for a while with Agisoft Photoscan. I used MATLAB to do the video processing. The script is here: readplanetvid.m. The main code bits include:
PlanetObj = VideoReader(videoname); %make a video object from an MP4 file
vidWidth = PlanetObj.Width; %get the width
vidHeight = PlanetObj.Height; %and height

mov = struct('cdata',zeros(vidHeight,vidWidth,3,'uint8'),...
'colormap',[]); %set up a MATLAB structure to contain the video

k = 1;
while hasFrame(PlanetObj)
mov(k).cdata = readFrame(PlanetObj); %pull out the frames one at a time from the MP4 object and put them in the mov
k = k+1;
end

step = floor(k/number_of_frames) %determine how many frames to skip each time to get the desired number
for i = 1:step:(k-1)
framepart = sprintf('_frame_%06d.png', i);
filename = strcat(foldername,'/',projectname,framepart);
imwrite(mov(i).cdata, filename) %easy to write the frame out as a png file
end

Video of a rock sample in my backyard. I took a short video of a piece of obsidian on a table with my Samsung J3. The nice thing about it was that I could gather a large number of views around the sample with relative ease. From that video, I extracted 150 frames, for example see below



I ran the files through the Agisoft Photoscan sequence of alignment (high), build dense cloud (medium), build mesh (medium), and build texture (medium). Here are a few screen captures of the result:

This one shows a somewhat complex background, but nicely indicates the path of the camera too.

And here it is with a trim to only show the sample on the table:

Structure from Motion using video frames; MATLAB for frame grabs and an example from satellite video

Introduction. Occasionally we and also we have gotten a question as to whether it is possible to use video frames as input for Structure from Motion models. This has certainly been done before with good success. For example, Yuichi Hayakawa did it starting with news video for a landslide triggered by the Kumamoto, Japan Earthquake in April 2016. Roman DiBiase showed me how he had done it video from helicopter and even performed topographic differencing with lidar for the Big Sur Landslide (e.g., NPR site and USGS site).

So, in a fit of procrastination, I decided to play around with the process myself. I was motivated originally by an idea from Andrea Donnellan and others at JPL to do topography from satellite video. They wrote a report entitled Gazing at the Solar System: Capturing the Evolution of Dunes, Faults, Volcanoes, and Ice from Space and I worked with Andrea and her team some on the problem.

Video to images. The main generic challenge for SfM from video is to extract the video frames and prepare them for the SfM. The SfM part is no different from what my group has been doing for a while with Agisoft Photoscan. I used MATLAB to do the video processing. The script is here: readplanetvid.m. The main code bits include:
PlanetObj = VideoReader(videoname); %make a video object from an MP4 file
vidWidth = PlanetObj.Width; %get the width
vidHeight = PlanetObj.Height; %and height

mov = struct('cdata',zeros(vidHeight,vidWidth,3,'uint8'),...
'colormap',[]); %set up a MATLAB structure to contain the video

k = 1;
while hasFrame(PlanetObj)
mov(k).cdata = readFrame(PlanetObj); %pull out the frames one at a time from the MP4 object and put them in the mov
k = k+1;
end

step = floor(k/number_of_frames) %determine how many frames to skip each time to get the desired number
for i = 1:step:(k-1)
framepart = sprintf('_frame_%06d.png', i);
filename = strcat(foldername,'/',projectname,framepart);
imwrite(mov(i).cdata, filename) %easy to write the frame out as a png file
end

Satellite video from Terra Bella. I have been watching the hi resolution satellite activity with great interest. Skybox had a few relatively high res (approx 1 m ground resolution) visible and near IR satellites with video capability. They were bought by Terra Bella (google) and then now are owned by Planet (who were just visiting us on the ASU campus last week and with whom we are building some collaborations). Some of the Terra Bella imagery is available on youtube. I grabbed one video of the Usak Mine in Turkey (used real player to convert youtube to mp4):

You can really see the parallax as the satellite moves over (not to mention the activity of the vehicles).

I ran my script on the mp4 and extracted 100 png frames. Here is an example:

I ran the files through the Agisoft Photoscan sequence of alignment (high), build dense cloud (medium), build mesh (medium), and build texture (medium). Here are a few screen captures of the result:
You can see the model and the camera positions. They are in the roughly correct arc, and relatively far away, but they should be much farther (orbit is approx 450 km).
Nice looking textured mesh. It is distorted, but not too bad, all things considered!
And, here is the point cloud in Cloud Compare.

What did we learn? We learned that the SfM from video is doable (see a future post from my backyard and phone video). Here is the Photoscan report on the Usak project. The geometry that is computed from the satellite video is not bad. Agisoft Photoscan does a pretty good job. We cannot get under the hood very easily to see more about the processing. I think that someone who knows more about computer vision than me would be able to comment as to the performance. I think that the main issue is probably the relatively low angular variation for the model.

Planet Team (2017). Planet Application Program Interface: In Space for Life on Earth. San Francisco, CA. https://api.planet.com.

Monday, June 5, 2017

Some new San Andreas Fault tour videos of 1 m bare earth hillshades

I have been preparing a lecture and I built some new simple videos flying along the San Andreas Fault. The videos are made by me flying along in Google Earth with 1 m hillshades produced from lidar topography data collected along the San Andreas Fault. The videos are on youtube in this play list: https://www.youtube.com/playlist?list=PLFfZSFyNZ_jZm86F1TsnYfuuYNef_Uh21. I also put the MP4s in this folder--they are numbered 1-8 from NW-SE.

The flights follow generally along the San Andreas Fault from Point Arena to the southern Carrizo Plain (Dragon's Back and Northern Elkhorn Hills):

The data were processed at www.opentopography.org and come from 3 really cool datasets:

Monday, May 22, 2017

One dimensional morphological modeling of transport and production- limited fault scarps

Over the years, I have maintained a steady obsession with fault scarps. For my Ph.D., I worked on a few aspects of fault-scarp development: Arrowsmith, J R., Pollard, D. D., and Rhodes, D. D., Hillslope development in areas of active tectonics, Journal of Geophysical Research, 101, B3, 6,255--6,275, 1996. Correction: Journal of Geophysical Research, 104, B1, 805, 1999. Since then, I have kept the work going along, mostly with teaching applications. In this blog post, I wanted to share some of the presentations and tools that are available to explore, learn about, perform one dimensional morphological modeling of transport and production-limited fault scarps.

A few definitions:

  • One dimensional--means elevation (H) as a function of distance along a profile (x).
  • Transport-limited--there is enough transportable material available for any erosion that comes from the application of the mass continuity equation. In this case, the transport capacity is equal to the sediment supply.
  • Production-limited--there is insufficient transportable material (regolith=material between topographic surface and top of bedrock) for erosion. In this case, the transport capacity exceeds the sediment supply locally.
  • Diffusion erosion--transport capacity is scaled by local slope and a constant k. The consequence of this transport rate choice and the application of continuity for transport-limited conditions yields a diffusion-like or heat-conduction-like behavior.

This presentation (PPT and PDF) provides a bit of a review of fault scarp research as I saw it mostly about 5-10 years ago. This PPT has two embedded movies which illustrate this basic behavior: PPT. Transport-limited scarp movie; Production-limited scarp movie

Transport-limited models:

This web page from my Computers in Earth and Space Exploration class lays out the main derivation and numerical implementation: Lecture 8: Exploring diffusion using Excel. This older page has some Matlab and Excel implementations of 1D transport-limited linear diffusion: Scarp diffusion exercise. Finally, here is a 2D version of transport-limited non-linear diffusion in a paper by Mattia de Michieli Vitturi and me: de Michieli Vitturi, M. and Arrowsmith, J R., Two dimensional nonlinear diffusive numerical simulation of geomorphic modifications to cinder cones, Earth Surface Processes and Landforms, doi:10.1002/esp.3423, 2013.

Production-limited models:

George Hilley significantly updated my original code and produced the Penck1D imlpementation in MATLAB: zip file. Here is an older version of the MATLAB (no gui): zip file.
The software is delicate in some ways so you may have to try it a few times! If it crashes, just start over. One important thing is that it works best if downhill is to the right.

Note in particular the user's manual we wrote in 2006: Hilley, G. E., and Arrowsmith, J R., Penck1d: Transport- and production-limited fault scarp simulation software, user's manual for software used at 2001 Geological Society of America Short-course: Tectonics and Topography: Crustal Deformation, Surficial Processes, and Landforms Cosponsored by GSA Structural Geology and Tectonics Division and taught by Dorothy Merritts and Roland Bürgmann.

Friday, May 5, 2017

Simple Topography Lectures at University of Geneva as part of CERG-C

I am visiting the University of Geneva as part of the CERG-C project. Professor Costanza Bonadonna is the leader of the activity which is a course on hazard and risk with an international group of students. They have been in Geneva for a few weeks for classroom work and starting tomorrow they go to Vulcano Island for practical experience. I am going to join.

I gave a lecture and a practical demonstration on the general topic of Topography, but trying to emphasize its fundamental value for science and applied value for hazard assessment. And, of course making the point that high resolution data are most useful.

Here are the pdfs to my lectures and demo:

Saturday, April 8, 2017

A pair of short courses on "Geoscience Investigations of Point Clouds" and "Advancing understanding of geomorphology with topographic analysis": mid June, 2017 at Potsdam University, Golm

Two short courses are scheduled for mid June at Potsdam University. The short courses are independent of each other; however, the topics are related and probably address a similar audience.

Geoscience investigations of point clouds, June 7-9, 2017. Instructors B. Bookhagen, R. Arrowsmith, M. Isenburg, C. Crosby.
This course will explore the acquisition, post-processing, and classification of point clouds derived from airborne and terrestrial lidar scanners and structure from motion (SfM) photogrammetry from drones. The course will take place at campus Golm (UP) and includes one day of field-data collection and two days of data post-processing and analysis.
The application is here: https://goo.gl/forms/NrRAcaASXPuseRs62. The course is sponsored by Geo-X.
Here is the flyer: PDF for more details.

Advancing understanding of geomorphology with topographic analysis emphasizing high resolution topography, June 12-15, 2017. Instructors R. Arrowsmith, W. Schwanghart, C. Crosby, B. Bookhagen.
This course will focus on advanced understanding of geomorphology with topographic analysis emphasizing high-resolution topography. The course will take place at campus Golm (UP) and includes theoretical background and analysis of digital topography using TopoToolbox in a Matlab environment. The course is sponsored by StRATEGy.
Here is the flyer: PDF for more details.