I found that if I made a Winform app that would load a WebBrowser control, I could load a streetview into the browser and update the heading (and pitch) as I moved my head. The Google Streetview browser control handles all the buffering of images so you can just update heading and it does all the work. The Winform app handles the GPS location data and allows me to track the head and body positions separately.
It actually surprised me how well it works without changing any of the code from the IR Camera Project using the IR Camera from the wiimote.
My Github.com Repo code is here: