Photogrammetry with Free & Open Source Software ( FOSS )

Update: More photogrammetry posts on this blog see below or click on “Photogrammetry” in the “Categories” list

Wikipedia defines photogrammetry as

“… the science of making measurements from photographs, especially for recovering the exact positions of surface points.” If you can recover enough of the surface points then it’s perfectly possible to recreate a complete 3D model of an object from 2D photographs.

( I first got interested in 3D scanning some years ago when I managed a project to develop some software to map and record human body shapes using a ( then quite expensive ) scanner made by a company called TC2.

The project was health related – http://www.bodyvolume.com/home. It’s an interesting idea with lots of health benefits which unfortunately still hasn’t really got into widespread use. I think that’s mainly due to the problem of scanning a human body – you can have any two of quick, cheap and accurate 🙂 )

Today there’s a slightly overwhelming variety of photogrammetry software options available and this post is a summary of the analysis that I went through when trying to select a suitable FOSS toolset to experiment with. I started with the following criteria to assess each option:

Must haves:

  1. Sensible build / install process. Ideally I should be able to fork a GitHub repository and build the software on any recent Linux distribution with any other dependencies able to be installed via standard packages. Life is too short for messing around with complex build processes and arcane library requirements.
  2. Good results / easy setup. It needs to work reasonably well with images taken with my point & shoot camera and without having to fiddle with parameters for every scan.
  3. FOSS. It needs to be FOSS and have no restrictions on the program’s use or on the use of the 3D models generated by the software.

Nice to have:

  1. GUI. A GUI sometimes makes getting started a bit easier but I’m also quite happy with command line tools especially if they can be wrapped in a shell script.
  2. Programming language. It would good to contribute something back to the project so ideally it would be written in a language that I’m familiar with. I don’t mind learning a new programming language but it’s time that could be better spent on other things.

I looked at the following options:

Python Photogrammetry Toolbox & PPT-GUI

The scope of the Python Photogrammetry Toolbox and associated tools seems to have evolved over the years and so is not the easiest thing to define. The simplest way that I found to try it was using the Archeos distribution. This installed fine on an old laptop that I had lying around and also on VirtualBox.

( I did briefly try installing the software on my normal Ubuntu based machine and it mostly worked. There were some problems with paths which would probably been fixable had I spent more time on it )

It runs fine and the GUI is easy to use. The only minor but annoying quirk is that it writes the output files to the /tmp/… directory which gets cleared on a reboot. You have to remember to copy the files to somewhere more sensible before exiting. As the name suggests it’s written in Python, it’s fully FOSS and there are no restrictions on the use of the models.

Archeos is based on Debian and also comes with many other related applications already installed. The team behind Archeos also have an interesting blog

openMVG & MVE

OpenMVG is

“ … a library for computer-vision scientists and especially targeted to the Multiple View Geometry community. It is designed to provide an easy access to the classical problem solvers in Multiple View Geometry and solve them accurately.”

openMVG doesn’t contain a module for generating dense reconstructions directly but the documentation suggests a few possibilities. I started to experiment with MVE and found it to be easy to use1

Both of these are written in C/C++ and are FOSS. I was able to clone and build from GitHub with no dramas other than installing a few extra packages.

There’s no GUI2,3 but it’s easy to put the commands into a shell script and run them that way. The outputs looked fine and I got good results fairly quickly from the models that I tried.

1It’s possible that MVE may do the whole job itself but I haven’t fully explored this yet

2There is a build of openMVG available with a GUI – https://github.com/open-anatomy/SfM_gui_for_openMVG/ However this seems to be a fork of openMVG and currently some way behind the original source. There’s a video tutorial using this GUI and it seems that it’s a way of selecting the command line options for the various tools, running them and displaying the output. For me that’s no real advantage over a shell script.

3MVE does contain a GUI for parts of the process but I haven’t investigated in any detail. See MVE build instructions for details.

Regard3D

Regard3D is essentially a GUI wrapper around several third party photogrammetry tools including openMVG and MVE. There are Windows, Linux and OSX versions but only the Windows and OSX versions have installers.

I tried the Windows version and it installed and ran fine. The GUI was easy to use and a useful guide to the options available to the various tools.

However I failed to get the Linux version to build despite spending a reasonable time trying. The main problems were with library versions, especially openCV.

The other drawback is that Regard3D uses a modified version of openMVG which means that there’s inevitably a delay in getting the latest version filtered through and it makes contributing changes back to openMVG more difficult.

OpenDroneMap

I didn’t do any detailed tests on this one because it seems to be more aimed at aerial photography but I included it for completeness.

The GitHub pahttp://opencv.org/ge describes it as:

OpenDroneMap is a tool to postprocess drone, balloon, kite, and street view data to geographic data including orthophotos, point clouds, & textured mesh. In the tradition of the Ship of Theseus, it was originally forked from qwesda/BundlerTools https://github.com/qwesda/BundlerTools.

I think that it’s command line based and written in Python.

Other Options

I also looked at a couple of other possibilities which didn’t quite meet the criteria above:

VisualFSM

VisualFSM has Windows, Linux and OSX versions and is “free for personal, non-profit or academic use”. The Windows version installs and runs OK although my results were not as good as those with openMVG/MVE

Unfortunately the Linux install instructions were lengthy, possibly out of date now and I failed to get them to work.

Online services

There are a number of online photogrammetry services, some are free as in beer, some are paid for, the Wikipedia page referenced below lists a few of these.

I have previously tried Autodesk’s free offering – 123D Catch. It worked reasonably well but gave no control over how the images were interpreted. However I notice that there are now mobile device apps as well – it was web only when I tried. These may offer more control.

Another drawback with these services is the licencing of the scans. For 123D Catch the scans remain your property but you give Autodesk:

“ … a world-wide, royalty-free, fully paid-up, perpetual, non-exclusive, transferable, and fully sublicensable (through multiple tiers) right and license (but not the obligation) to reproduce, distribute, redistribute, modify, translate, adapt, prepare derivative works of, display, perform (each publicly or otherwise) and otherwise use all or part of Your Content, by any and all means and through any media and formats now known or hereafter discovered”

I suspect the other services have similar terms. Of course there is also the possibility that any free services may be discontinued or may have to be paid for at some point in the future.

Conclusion

For me the obvious solution was openMVG/MVE. They’re completely open source, build easily and provide good outputs with minimum configuration changes. There’s no GUI but, to be honest, it was trivial to write some shell scripts to manage the whole process.
In a future post I’ll document how I built these tools and how I set up the directories and shell scripts to manage the model creation process.

For now here are two sceenshots of a stone figure on my neighbour’s wall ( original photographs taken with my Canon IXUS 70 point & shoot camera )

Other Notes

Wikipedia also has a long list of photogrammetry software but this includes some very expensive commercial offerings. There’s an older ( 2015 ) summary of the cheap / free options here – https://ryanfb.github.io/etc/2015/01/23/photogrammetry_software_roundup.html

This entry was posted in FOSS, Photogrammetry. Bookmark the permalink.

10 Responses to Photogrammetry with Free & Open Source Software ( FOSS )

  1. Pingback: Basic Operation of openMVG/MVE | The Retired Engineer

  2. Bill Wolf says:

    Thanks for this update, I’ve been out of Photogrammetry for a while, and found that 123d catch is gone, replaced with ReMake, and CMPMVS is now CapturingReality with a hefty price tag.
    You saved me a boatload of time, thanks for doing the research!

    Like

    • Glad to be of help 🙂 As you’ve found out, online options are tempting but they do come and go at the whim of the supplier and commercial software can be very expensive.

      I’ve had good results with openMVG & MVE and had very quick and informative replies to any questions that I posted on GitHub.

      Like

      • Bill Wolf says:

        I’ve only just started looking, but is MVE just generating a dense point cloud, or is it building the mesh? What do you think of Regard3D? I’ve worked with VisualSFM and the workflow associated with it, and at this time it would be nice to just have a neat interface to the whole mess.
        If you haven’t tried ReMake, give it a shot. I sent the same photos I sent to 123d catch over a year ago, and it makes MUCH better models now. Creepy impressive what it can do for free with just 50 pics.

        Like

    • MVE generates both the dense point cloud and the mesh itself. It also has a mesh clean up tool which is pretty effective.

      Regard3D works fine. To be honest I would have probably persevered with it if I had managed to get it to build on Linux. However after adding my camera to the openMVG database I did find and fix a small bug with the camera detection code which would have been a lot more difficult with Regard3D.

      User interfaces are tricky things to get just right. I’m comfortable with the command line and I’ve put together some scripts and a workflow for openMVG/MVE – https://theretiredengineer.wordpress.com/2017/07/02/basic-operation-of-openmvgmve/ Note that these tools have many more command line options and I’ve only just scratched the surface so far. My plan is to add more controls in the scripts as I come to need them.

      Thanks for the tip-off on ReMake, I’ll run some comparisons with openMVG/MVE in due course.

      Like

      • Bill Wolf says:

        Thanks again for the info, I may have to try your workflow. After adding my camera to the database, it still failed to reconstruct anything meaningful from my images, but ReMake used the same images and made a model so detailed you could pass off almost any view as a photo.

        Like

  3. What version of openMVG are you using and on what platform? The bug that I fixed on camera selection could cause these issues – ( https://github.com/openMVG/openMVG/issues/794 ) The fix is in the latest Master version on GitHub but may not be in the latest version of Regard3D for example.

    One quick check is to replace the current “sensor_width_camera_database.txt” file with one containing just your camera. This should eliminate the possibility of this bug being the cause.

    Like

    • Bill Wolf says:

      Turns out the problem was the pictures were taken with a really bad phone camera app that was putting out pictures with strange sizes with no EXIF data at all. Couldn’t tell you how ReMake is getting it to work.
      Also check out COLMAP just for fun if you haven’t. It isn’t FOSS, but I had never heard of it before and it pretty much blows every other free package out of the water. My computer is a decade old and can’t really handle it, but you can build it on Linux if you have a rig set up to do this kind of work.

      Like

  4. Pingback: 3D Foot Scanning | The Smell of Molten Projects in the Morning

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.