8 replies to this topic

[Hans van der Maarel]

I've been given an Excel file with coordinates written like this:
53.20.00.86N05.13.08.40E

I assume that's 53 degrees 20' 0.86" North by 5 degrees 13' 8.40" East, in WGS84 lat/lon.

However, combining this with other data shows errors varying between ~30 and ~200 meters, which is unacceptable.

So I'm wondering about the last 2 numbers for each coordinate. Should I read 08.40 as 8.4 seconds or as 8 seconds + 4/60th of a second? Then again, this would not cause errors in that range.

Any thoughts on this? I'm guessing there's something wrong with the source data, but I want to make sure it's not something in the conversion process.

[benbakelaar]

Maybe its not WGS84? Could coordinates in one projection vary as much as 20-300 meters with WGS84 coordinates?

[Martin Gamache]

Maybe its not WGS84? Could coordinates in one projection vary as much as 20-300 meters with WGS84 coordinates?

<{POST_SNAPBACK}>

If it is a datum error, the shift should be fairly consistent I would expect.

How was the data collected, i.e. where do the points come from. Could this be an issue with some GPS data being colected in difficult terrain or with an older device?

[frax]

I would guess the latter, that sounds more consistent (8 secs + 4/60 of a sec)

[Hans van der Maarel]

Martin,

The errors aren't consistent enough to be attributable to something as straightforward as a datum shift or the 8.4 vs 8 + 40/60th (just now noticing I had that wrong in my original post) thing, so there's probabely some acquisition errors in there.

All I know is that the data is captured by our Hydrographic Service, no word on method or file structure.

[Dennis McClendon]

Are there any M or S values greater than 60? Is it possible that those are also in decimal (hundredths) rather than sixtieths?

[Hans van der Maarel]

Are there any M or S values greater than 60?  Is it possible that those are also in decimal (hundredths) rather than sixtieths?

<{POST_SNAPBACK}>

I hadn't gotten around to going through the 6000-line Excel file... I didn't notice any 60+ values in my quick glances.

[Dennis McClendon]

Well, glancing at the first 100 or so should be enough.

What if you just truncate the hundredths? At the scale you're working, would seconds be sufficient accuracy? Each second is only 20-30 meters or so.

[Hans van der Maarel]

Well, we've established that the coordinates are actually in WGS and the last set of 2 digits is the hundredths of seconds. Turns out my FME process was introducing a slight error (a division by 100 is not the same as a division by 100.0), but still there's some differences. Client is now deciding whether or not it's acceptable.