Category Archives: Longform

Google Cardboard 3D Viewer

Hardware

As well as a suitable mobile phone then you’ll need a headset of some sort. Google still sell the original cardboard versions but there are hundreds of different ones available. After a quick look round I got myself a second hand Homido headset for the grand sum of £10 including p&p from Ebay.

It has the same cheapo plastic lenses as the Google cardboard version but has a much better strap and also some crude, but quite effective, controls for adjusting the lens and phone positions. There’s also the usual single button control which just moves an arm to make contact with the screen so acting like a single button mouse.

Note that Google Cardboard only has 3 Degrees of Freedom and will only track head movements. It won’t track your movements if you start walking around. The HTC Vive has 6 Degrees of Freedom.

Software

The best starting point for the software is Google’s Cardboard app. This has a few examples built in but is also used by other 3rd party applications as well. On first installation it asks for some details of the headset dimensions or you can scan a QR code on the headset if available which does the same thing. ( You can see the QR code on the front flap in the photo above )

Applications

Examples

There won’t be many screenshots in this post because the side-by-side images that are combined to make the 3D view don’t really make sense in 2D. This is the intro screen to the Cardboard app. The small circle just above the “Exhibit” text of the highlighted option is in effect a cursor. It stays in the centre of the vision and allows you to highlight options by hovering over them. The button control on the headset then works the same as a mouse button and clicking it selects that option. It’s basic but does the job.

The display is not too bad but suffers a little bit from the screen-door effect which is to be expected I suppose. It’s definitely worth fiddling with the headset adjustments because these can make a big difference to the quality of the image. It’s certainly as good as the original Oculus but nowhere near as good as the HTC Vive.

There are a few built-in examples:

Photo Sphere – more on this below
Exhibit – Displays a 3D model that can be rotated by turning your head. This is quite effective and a neat way of viewing the models. Clicking the button cycles on to the next model
Arctic Journey – This is a low poly rendering of an Arctic landscape with a few options allowing you to navigate through it. A good demo of what can be achieved.

360° Photo Viewer

The Photo Sphere option above scans the phone for suitable 360° images and displays them on the headset and allows you to turn your head and see the image all round. Disappointingly there are no images included as default but it’s fairly easy to find a suitable one online. I showed this application to my non-technical consultant ( my wife! ) and she was quite impressed. Interestingly the photo that I’d found was from the top of a cliff overlooking a fjord and, as she’s not too keen on heights, she found it a little too realistic when looking down at the water 🙂

The next question of course was “How can I take my own 360° images?” After a bit of messing around the answer was to be found in the Google Street View app which has an option for creating Photo Spheres. Obviously you’re expected to upload these to Street View but the images can easily be extracted from the phone if you wish ( from the “/Internal shared storage/Pictures/panoramas” directory on my phone ). A downsized example from one of my walks on the Brecon Beacons is shown below.

The original was 10240 x 5120 pixels

Taking the images is really easy, click the “create” option and follow the instructions. The camera decides where the pictures are to be taken and you just have to line up the dots – you don’t even have to press the shutter. There are the usual stitching problems with things like fast moving clouds but it does a pretty good job of managing the exposure setting across the image. One point to watch that I found is that when taking the pictures pointing towards the ground it’s important to hold the phone at arm’s length otherwise you end up with a pair of disembodied feet in the final image 🙂

An alternative would be to use a dedicated 360° camera. These are now getting to be more popular but still seem to be around the £400 price point which is a little steep for experimenting. Once second hand ones get down to £100 on Ebay then I may splash out.

I haven’t yet tried taking an image inside a building but this is certainly on the post-COVID to-do list.

360° Videos

As far as I can see 360° cameras are mainly used for taking video and there are plenty of these on YouTube. Rollercoaster rides for example:

You can look round using the desktop web version but the VR headset view is more spectacular. You switch by clicking on the “mask” icon. Of course you don’t get the full stomach churning experience, you’ll need to build your own hydraulic platform for that!

However viewing 360° videos on YouTube shows up one of the shortcomings of this interface which is that it’s not possible to navigate the Android YouTube app with the VR goggles on. So you first have to find the video with the goggles off, then switch to 3D view and then put the goggles on. It’s OK for just one video but pretty quickly gets tiresome for further videos.

Desktop Apps

Using VR goggles I wondered if it was possible to create a huge virtual display similar to sitting inside a sphere with the screens projected onto the wall. There is an Android app called Fulldive VR which sort of gives a big virtual desktop but I found it difficult to navigate with just the in-vision cursor and single button press. From what I’ve read it’s possible to integrate more complex Bluetooth based peripherals like keyboards and multi button mice with Google Cardboard but the results can be hit or miss.

Another option was a web browser with multiple windows. There are a few of these and VR Browser seemed to get good reviews. From my testing it works fine but entering URLs one character at a time using the onscreen keyboard and button clicks was painful to say the least. Also the screen resolution is not quite good enough to read text. However, with a bit of effort, it would be possible to fill the whole sphere with browser windows should you so wish! Playing back movies in a YouTube browser window was very jittery though.

Sadly all that means that we probably won’t be getting a Johnny Mnemonic type interface with my budget setup 🙂 This is a shame because I do like his data gloves.

( The novel Snow Crash by Neal Stephenson also has similar themes and is worth a read. )

One final approach that I tried was to run the interface on my desktop PC and connect via VNC from the phone. There is naturally an app for this called VR Remote Desktop Free which will connect via VNC to a server and show the window in the VR display ( see details below ). As an extra it can also display the phone camera view as a background so you can see what’s around you. Terminator vision anyone!

This approach came the closest to actually being useful but suffered from one major bug which scuppers it completely. Because the sensors in the phone are not really designed for this application then there needs to be some clever software to stop the display drifting over time. Unfortunately this app doesn’t implement that correction so the VNC window slowly drifts left or right. There’s a Reddit thread complaining about this from 5 years ago so it’s not likely to be fixed anytime soon 😦

Playing video over this connection was also jittery which is mainly why I didn’t pursue it too much. There are some interesting comments and suggestions here about how this could be sped up but it’s not a simple process.

VNC Setup

There are many VCN server options for Linux but I used X11VNC as it was already installed and it’s easy to use. It also has the advantage that it can VNC a single window, e.g. browser, rather than the whole desktop. The process was:

Run the xwininfo command to get the id of the window to be used
Start X11VNC using the -sid option and using the window id obtained in the previous step
Connect from the phone using VR Remote Desktop Free

Note: If you really want to terrorise the local neighbourhood with Terminator Vision then I recommend this blog post on how to do it with Microsoft’s HoloLens 🙂

3D Model Viewer

The “Exhibit” demo in the Google Cardboard app is quite a good way of examining a 3D model. However it seems to be limited to the half dozen models that are built in to the app and there’s no way of adding any more. Although there are plenty of 3D model viewers on Google Play I only found one free one that supports a VR view – 3D Model Viewer. This works very well but ( there’s always a but! ) it doesn’t seem to display the object colours, it only shows the mesh. This is a shame because otherwise it works very well. Again there are requests on the app page for this going back a few years so I won’t hold my breath.

The Sketchfab app purports to show models in VR view but I never got it to work properly. I think that the problem is that the models need to be relatively low-res and mine are not.

The last idea that I had was to see how easy it would be to immerse myself into a 3D mapped landscape like this model of the Stubaital in Austria:

It would be similar to a Photo Sphere but generated from a model. Ideally you could also wander about a bit on the valley floor to see different views.

Regrettably, but probably not surprisingly, I didn’t get very far with this idea. I think that at a high level it will involve importing the terrain into a game engine that supports VR then positioning the character and defining the movement limits. As an example the open source game engine Godot has had VR support for a few years now so it should be fairly mature. However I decided not to venture down that particular rabbit hole for the time being 🙂

Conclusion

Overall this was a very interesting few days investigation and well worth the £10 entry fee. A decade ago when I first saw the Oculus Rift I had the feeling that, other than for video gaming, this was a solution looking for a problem. I think that, for once, I was right in my predictions. The non-gaming applications seem to be going the way of augmented reality rather than full VR immersion.

I’m sure that the lack of a smooth interface between the desktop and the VR goggles is a problem. For example my ongoing but pandemic-delayed project on taking photos from trig points would greatly benefit from a full 360° VR compatible views but I have no idea on how the interface might work alongside the desktop version.

However, on a positive note, the Photo Sphere app is fascinating and I certainly want to try taking more of these, especially an image of the inside of a building. I probably also need to keep an eye on Ebay for second hand 360° cameras 🙂

Online Learning

Technical / Programming

Because of the origin of the internet then I suppose that it’s logical that there are many online resources and advice on technical or programming matters. However, like real life, the quality of these vary from the excellent to the completely wrong 🙂 Although I regularly get ad-hoc technical information online I have also done a couple of free courses:

Udacity ” … began as an experiment in online learning, when Stanford instructors Sebastian Thrun and Peter Norvig elected to offer their “Introduction to Artificial Intelligence” course online to anyone, for free. Over 160,000 students in more than 190 countries enrolled.”

My aim here was to learn how to produce an app for Android devices. This was partly to give me some background as a software project manager and partly for dabbling in my own time. I did three courses:

How to Use Git and GitHub: This used to be course ud775 which not available now. It’s been replaced by course ud123.
Java Basics: This used to course cs046 which is not available now. It’s been replaced by course ud282
Android Basics: This is a series of courses which forms part of the Android Basics Nanodegree

All courses were free, you only need to pay if you want some actual formal certification. I completed 1 & 2 and stopped 3 after discussion of single page apps as I wasn’t interested in anything more complex. The pace was maybe a little slow in parts but overall it was great. The material was well presented and the comprehensive course notes could be downloaded. The only minor issue was that Google tends to regularly change the user interface of their Android Dev Studio and occasionally the notes hadn’t quite caught up with the latest version but it was fairly easy to work it out.

It may come as a surprise but YouTube is not just for cat videos 🙂 There’s also some rather good technical content.

My aim here was to understand some of the concepts that I kept coming across when reading about photogrammetry. Fortunately The University of Central Florida has uploaded a series of undergraduate lectures on computer vision and these covered the topics that I wanted to learn about. The presentation is fairly basic with a single camera for the lecturer and the slides overlaid on the video. It was a bit of a flashback to sitting in lecture theatres 40 years ago and reminded me of Mortimer J. Adler’s quote:

A lecture has been well described as the process whereby the notes of the teacher become the notes of the student without passing through the mind of either.

However I listened to half a dozen hour long lectures and I picked up enough information to greatly improve my understanding of what’s going on in the photogrammetry process. There was also some course work to go with these lectures and it would have been even more useful if this was available for download.

Interestingly I subsequently found a rather good YouTube channel ( Computerphile ) which explained the Sobel Edge Detection process in less than 8 minutes rather than the hour taken by the UCF lecture ( The first comment on the YouTube video reads “My computer vision professor should be fired…” 🙂 )

Foreign Languages

Despite the regular hand wringing about the lack of foreign language teaching at secondary school I’ve found that evening classes on this subject have always been quite popular. Strangely during the pandemic when travel was restricted language learning also spiked. I’ve attempted a number of languages over the years using a few different methods:

French: After doing various French evening classes I decided to up my game and sit a French AS level. I used a paid-for course from a company called Oxford Open Learning which was a more traditional distance learning course rather than being purely online. The course materials were good and thorough and there was plenty of support from a remote tutor. I made sure that the course used the same exam board as the local school which is where I sat the exam. ( And yes I passed – with an ‘A’ grade thank you 🙂 )

Italian: Following my success with the French I decided to try Italian. For this I used the Open University course – “L195 Andante: Beginners’ Italian” This was a bit of a hybrid with mainly traditional course materials which were very good and some online work using the OU’s audio conferencing software. There were also occasional in person tutorials with the local tutor which were excellent. The exam at the end was done via the audio conferencing system. ( And yes I passed this as well 🙂 )

Both of these needed a lot of self-motivation to complete but were very good and I ended up with actual, useful paper qualifications. ( I had maybe thought of going on and doing a modern languages degree after retirement but for various reasons that didn’t happen. )

Fast forward a few years and we see the rise of the language learning app. Duolingo is one of the popular ones but obviously alternatives are available – there’s an interesting discussion here. All comments below refer to Duolingo’s free version but I think that they should be applicable to the paid plan as well.

German: I’d done some German over the years and I was looking for a revision course to pull together the various different parts. It was easy to get going with the app and there was a good spread of vocabulary subjects. I liked the regular vocabulary testing and the gamification ( daily streak, rewards etc ) was actually quite useful. The downside was that I felt that there was a lack of explanation of grammar. This is not good news for a German course (!) but fortunately I had a few text books that covered the gaps.

Finnish: This was a bit of an experiment as, although we’ve been to Finland a few times, there’s no need to learn Finnish as they all speak good English. Again the app was good for learning and testing vocabulary but it was a bit hopeless for grammar. Unfortunately this is even more important for Finnish than for German and, unsurprisingly, our local library doesn’t stock any suitable textbooks. The other problem was that the course appeared to be run by volunteer admins for whom English was a second language which meant that the English translations were sometimes a little off. The admins were occasionally reluctant, sometimes to the point of obstinacy, to accept alternative suggestions.

Welsh: This was an interesting exercise as it was my wife that was learning Welsh and I could review the app as a native speaker. Again it was generally fine but I had two comments. Firstly most good Welsh courses will specify whether they’re targeted towards North Wales or South Wales Welsh dialects. They’re obviously the same language and understanding one or the other isn’t a problem but there’s sufficient different vocabulary to merit having two different versions of the course. Duolingo seemed to mix the two which will be very confusing for a learner. Secondly Welsh grammar has some very complicated consonant mutations [ 1, 2 ] and you’re never going to work them out without an explanation. Fortunately my wife was doing an evening class with a very good teacher so she could fill in the gaps from that course.

In summary I found Duolingo excellent for revision but had serious shortcomings if you want to just learn using the app. There are grammar notes are available but they’re quite hard to find IMO. However help is at hand and there’s an independent website that hosts all grammar notes – https://duome.eu/tips/en. I wish I’d known about this when doing the Finnish course 😦

Music

Apps for learning music also seem to be increasing in popularity. Naturally there are lots of different types available but below are two that I’ve tried:

We recently bought a new Yamaha electric piano ( a Clavinova 735 ). With it came 3 months free subscription to Yamaha’s Flowkey app. I gave it a go and it’s a well thought out and implemented app with a range of exercises and pieces to play. However there are two main problems which put me off and I only used it for a couple of weeks

There’s no traditional sheet music view. There’s only a single line scrolling view which I found difficult to follow. Maybe this wouldn’t be an issue if you’re starting from scratch but I struggled with it.
The “learning” mode pauses the scrolling until you play all the right notes. This is not always the best method of learning music. Often it’s better to get the flow of the whole piece first before worrying about getting all the right notes.

There’s a full review here.

This is similar to Flowkey and it has a paid plan which includes some feedback from actual piano teachers. However the free Pianote YouTube channel videos are excellent, well presented and provide a lot of useful tips along with downloadable sheet music.

Conclusion

In my experience learning anything beyond the basics is hard but some of the resources that I’ve described have certainly helped me get started. I’ve found that, lthough personal learning preferences are important, the key ingredient is motivation. Without this it doesn’t matter how good the course is, it will always be a struggle.

( I’m particularly irritated by YouTube ads showing how easy it is to learn to play the piano. You may be able to learn some pieces by rote but unless you’re really musically talented it’s going to take a lot of dedicated work and practice to be able to just pick up a new piece of music and make a reasonable attempt at it. )

One other point that I’ve noticed is that online video conferencing is still generally quite patchy. There are always issues with the sound, the lighting or the bandwidth making things glitchy. I think there’s plenty of room in this field to produce something a lot better but I suppose that the problem is people don’t want to pay for it.

The Retired Engineer is lurking at the rear right, in the red checked shirt!

In the end though however good you make the video conference it’s just not as good as being there in person. For reasons that I won’t bore you with I’ve ended up running a Welsh learners conversation group. We used to meet in the pub but for the past year have switched to Zoom ( for obvious reasons! ) and it’s just not as good. Roll on the time when we can get back to the pub 🙂