Bivouac.com   Discussion Page   Home     Help   Index     Login
Trouble Ticket Discussion - GMap Problem #5625
Back To Discussion List Written: 2020.07.01 by: Robin Tivy

GMap had a problem and it took a lot of work to resolve it. Meanwhile various trouble tickets piled up. With problems like that, I would like to reply to everyone who is interested, and post any updates. To do that I set up this pseudo "discussion" and then subscribed everybody to it that had reported a problem. Then when the problem is resolved, I'll do a posting and every subscriber will be informed.

I'll put link to this discussion in the trouble tickets. If you see a trouble ticket discussion that you want to follow, just subscribe, and you'll be informed when it is resolved.

The problem in this case is that somehow on the weekend, GMap quit working. It would just hang and then eventually generate an erroneous message about missing headers.


Comments

#6726 - 2020.07.01 Greg Jones - Great work!
Sounds like a lot of energy went into the fix. Good job!

#6725 - 2020.07.01 Robin Tivy - Working Again
It is now working. Something had gone wrong when Joseph moved code from development to production.

#6724 - 2020.07.01 Robin Tivy - Seems Broken again!!
I just tried to run Gmap and it seems broken again.

#6723 - 2020.07.01 Robin Tivy - Problem Resolved I thought
This GMap problem was a major challenge in remembering how inter-server communications work. After two days of being on the phone to various advisors, we fixed the problem.

Thank you for reporting it. Just for your interest, I can tell you a bit about the problem. Here's the full story, like a TRIP report.

 - I came home Sunday night after a failed attempt to climb Peers Peak.
 - There was a trouble ticket in my inbox sunday night
 - I stumbled downstairs, and began the process of remembering everything about how Gmap works.
 - I found my test document of GMap tests. Soon determined that many of my tests were failing. Many of the tests were out of date.
 - phoned Joseph and updated my tests
 - now we can see what works and what doesn't work
 - figured out that the root problem was that his server could not read the overlay files (.ssv files)
 - would not read even simple text file from my server.
 - sent email to Joseph. He always answers right away, but for some reason he didn't
 - I waited a day, then phoned him. I was so glad to hear his voice. Turns out he had sent a bunch of email replies to me which disappeared into deep space. That's why I didn't hear from him.
 - So now joseph started testing trying to access .ssv files from his machine.
 - Joseph got a message that it was a firewall problem from cURL. After much discussion, we didn't believe that message.
 - tried to access a simple 3 line file called test1.ssv from various workstation machines. This request would generate an error in any browser.
 - discovered that browsers had no problem requesting .txt files, but would not work with .ssv.
 - Bill and I tested the ISAPI URL rewrite, which is the real thing that generates the phantom .ssv files on the fly. It delivered the contents of the file to Bill's browser no problem. So why not the 3 line ssv file?
 - inspected the MIME types in the IIS server and discovered that .ssv was set to text/XML. I can't ever remember setting that. So that explained why the browsers would show an error, because test1.ssv wasn't a valid xml file.
 - So I changed MIME type to "text/plain". Still didn't work with Google Chrome till browser cache was cleared. Now it displayed properly! https://bivouac.com/test1.ssv
 - Now we proved that accessing the file worked on all the browsers. Fixing the mIME was correct, but that wasn't the problem. Josephs SERVER still could not access even the simple test1.ssv file.
 - But Joseph COULD access the file from his server using the wget command on the Linux command line.
 - Joseph thought it was "handshaking" for the HTTP protocol. Or the DNS. I couldn't see how DNS would have anything to do with it.
 - Joseph was thinking maybe he had to upgrade his OS.
 - I was more and more suspicious of his ISP. I thought that Joseph should contact the elusive technical support at the ISP that hosts MappingSupport. Joseph and I agreed that somehow the problem was in his part of the world.
 - Joseph got cURL to generate a more verbose error message, which reported clearly that Port 443 was blocked at Bivouac.com. He also sent me a 2 page "brain dump" of things he was thinking.
 - I noticed in the error message a strange IP address that was not Bivouac and not Mapping Support. So I suggested maybe it was a firewall operated by his ISP
 - Turns out that address was a machine at BlueHost, his ISP! Somehow the DNS server that his server was using was translating Bivouac.com into this strange server, which then reported that port 443 was blocked. So Joseph then updated his server so it used a DNS resolver at Google instead of the one provided by his ISP.
 - AND SUDDENLY EVERYTHING WORKED. So the problem had nothing to do with MIME types, it was a corrupt DNS server at his host.

  META CONCLUSION
 I really hate working on these things. The worst thing is you can't remember how the stuff works. Or maybe you never knew. how does MIME work? What tools are there to access files on servers? What firewalls exist? Every phone call we blundered around with various concepts. It was like trying to manage a moon launch from a care home where we'll all end up. None of us were really sure how anything worked, but everybody had lots of partial knowledge of a lot of things.