Take the 2-minute tour ×
Mathematics Stack Exchange is a question and answer site for people studying math at any level and professionals in related fields. It's 100% free, no registration required.

Is it possible to quantify the maximum error expected when using the equirectangular distance approximation for distance on the globe.

λ = longitude

φ = latitude

x = Δλ.cos(avg φ)

y = Δφ

d = R.√(x² + y²)

I am guessing the absolute maximum would be the distance between (lat,lon) (0,0) and (90,180), but I am more interested in quantifying the maximum expected error when abs(Δλ) + abs(Δφ) < some nomimal constant (like 1 degree).

share|improve this question
add comment

1 Answer 1

I was looking for the same answer. The error depends on the distance, the bearing between the two points, and the latitude. So since there's nothing here, I decided to do a little empirical search. Rather than using the average latitude, I will use the first latitude - in my use case I'm computing many small distance from some point on an embedded device, and by using cost(lat1) I can avoid recomputing the cosine, and do all computation in integers using fixed point arithmetic.

My empirical test considered a set of latitudes, bearings and distances. For every bucket, I compute 50K uniformly distributed random examples. I compute the error as (great circle distance / approximated distance). In the following tables, the minimum such factor, the maximum and the standard deviation is shown:

distance (km) vs lat:
max factor:
[[     0.          10.          20.          30.          40.          50.          60.          70.          80.          90.     ]
 [     0.01         1.           1.           1.           1.           1.           1.           1.           1.           1.14428]
 [     0.1          1.           1.           1.           1.           1.           1.00001      1.00001      1.00002      1.81028]
 [     1.           1.00001      1.00001      1.00002      1.00003      1.00004      1.00005      1.00008      1.00017      2.69966]
 [     3.           1.00002      1.00003      1.00005      1.00008      1.00011      1.00016      1.00025      1.00051      3.0191 ]
 [    10.           1.00005      1.00011      1.00017      1.00025      1.00036      1.00052      1.00083      1.00171      3.10754]
 [    30.           1.00016      1.00033      1.00052      1.00076      1.00108      1.00157      1.0025       1.00519      3.25157]
 [   100.           1.00054      1.00111      1.00175      1.00255      1.00362      1.00532      1.00841      1.018        3.19962]
 [  1000.           1.00635      1.01235      1.01934      1.02844      1.04142      1.06335      1.11315      1.42766      3.25602]
 [ 10000.           2.36138      2.55231      2.70347      2.87681      2.95224      3.10016      3.13947      3.19104      3.15955]
 [ 20000.           2.20339      2.08726      1.94497      1.7517       1.5221       1.28392      1.13302      1.06562      1.01239]]
min factor:
[[     0.          10.          20.          30.          40.          50.          60.          70.          80.          90.     ]
 [     0.01         1.           1.           1.           1.           1.           1.           1.           1.           0.85811]
 [     0.1          1.           1.           1.           1.           1.           0.99999      0.99999      0.99998      0.80262]
 [     1.           0.99999      0.99999      0.99998      0.99997      0.99996      0.99995      0.99992      0.99983      0.79752]
 [     3.           0.99998      0.99997      0.99995      0.99992      0.99989      0.99984      0.99975      0.99949      0.7965 ]
 [    10.           0.99995      0.99989      0.99983      0.99975      0.99964      0.99948      0.99918      0.99831      0.79641]
 [    30.           0.99984      0.99967      0.99948      0.99925      0.99893      0.99844      0.99753      0.99499      0.79633]
 [   100.           0.99948      0.99891      0.99829      0.9975       0.99647      0.99488      0.99198      0.98384      0.79632]
 [  1000.           0.99566      0.99025      0.98425      0.97731      0.96849      0.95569      0.93444      0.88781      0.79632]
 [ 10000.           0.98929      0.9609       0.92391      0.88504      0.84837      0.81731      0.80094      0.79732      0.79638]
 [ 20000.           0.9893       0.96028      0.91911      0.87388      0.83463      0.81067      0.80169      0.80184      0.84316]]
std deviation
[[     0.          10.          20.          30.          40.          50.          60.          70.          80.          90.     ]
 [     0.01         0.           0.           0.           0.           0.           0.           0.           0.           0.00006]
 [     0.1          0.           0.           0.           0.           0.           0.           0.           0.           0.00042]
 [     1.           0.           0.           0.           0.           0.           0.           0.00001      0.00001      0.00158]
 [     3.           0.           0.           0.           0.00001      0.00001      0.00001      0.00002      0.00003      0.00282]
 [    10.           0.           0.00001      0.00001      0.00002      0.00003      0.00004      0.00006      0.0001       0.00501]
 [    30.           0.00001      0.00002      0.00004      0.00006      0.00008      0.00012      0.00018      0.00032      0.00895]
 [   100.           0.00003      0.00007      0.00012      0.00019      0.00026      0.00038      0.00057      0.00104      0.01616]
 [  1000.           0.00025      0.00065      0.00111      0.00167      0.00239      0.00345      0.00534      0.01073      0.04507]
 [ 10000.           0.01704      0.02979      0.04012      0.04654      0.04911      0.04796      0.04474      0.03961      0.02325]
 [ 20000.           0.04231      0.03707      0.02943      0.02307      0.02209      0.02554      0.02726      0.0225       0.01033]]

bearing vs lat, distance in range 0..1000.0000km:
max factor:
[[   0.        10.        20.        30.        40.        50.        60.        70.        80.        90.     ]
 [   0.         1.00031    1.00061    1.00101    1.00153    1.00242    1.00421    1.01039    1.2449     3.2419 ]
 [  15.         1.00238    1.0048     1.00768    1.01161    1.01839    1.03044    1.0688     1.42566    2.13819]
 [  30.         1.00512    1.00971    1.01585    1.02372    1.03584    1.05781    1.11051    1.39268    1.56511]
 [  45.         1.00634    1.01231    1.01922    1.02827    1.04142    1.06296    1.1115     1.27943    1.29318]
 [  60.         1.00622    1.01229    1.01908    1.02809    1.04071    1.05891    1.0948     1.1373     1.13721]
 [  75.         1.00482    1.00954    1.01415    1.02036    1.02746    1.03688    1.04607    1.04718    1.04708]
 [  90.         1.0016     1.0029     1.00411    1.00482    1.00513    1.00514    1.00515    1.00514    1.00512]
 [ 105.         1.00044    0.99979    0.99959    0.99934    0.99908    0.99866    0.99786    0.99664    0.99999]
 [ 120.         1.00093    0.99952    0.99903    0.99843    0.99769    0.99679    0.99536    0.99259    0.99998]
 [ 135.         1.00101    0.99958    0.9991     0.99857    0.99803    0.99716    0.99602    0.99373    0.99999]
 [ 150.         1.00091    0.99981    0.99961    0.99937    0.99905    0.99872    0.99811    0.99702    1.     ]
 [ 165.         1.00049    0.99997    0.99995    0.99992    0.99988    0.99984    0.99977    0.99963    1.     ]
 [ 180.         1.00006    1.         1.         1.         1.         1.         1.         1.         1.     ]
 [ 195.         1.00047    0.99997    0.99995    0.99992    0.99987    0.99982    0.99977    0.99964    1.     ]
 [ 210.         1.00091    0.99981    0.99959    0.99936    0.9991     0.99875    0.99815    0.99704    1.     ]
 [ 225.         1.001      0.99957    0.99915    0.99861    0.99803    0.99713    0.99591    0.99385    0.99999]
 [ 240.         1.00089    0.99954    0.99899    0.99845    0.99777    0.99684    0.99539    0.99242    0.99998]
 [ 255.         1.00044    0.99979    0.99958    0.99935    0.99907    0.99859    0.99807    0.99638    0.99999]
 [ 270.         1.00154    1.00291    1.00404    1.00494    1.00514    1.00515    1.00514    1.00515    1.00514]
 [ 285.         1.00483    1.0093     1.01428    1.02018    1.02734    1.03678    1.04591    1.04722    1.04724]
 [ 300.         1.00631    1.01227    1.01915    1.02791    1.04011    1.05906    1.0939     1.13728    1.13749]
 [ 315.         1.00635    1.01235    1.01934    1.02844    1.04115    1.06335    1.11315    1.27722    1.29346]
 [ 330.         1.00506    1.01004    1.01581    1.02362    1.03619    1.05676    1.11086    1.38735    1.56542]
 [ 345.         1.00245    1.00475    1.00754    1.0118     1.01827    1.03074    1.07095    1.42766    2.13419]
 [ 360.         1.00031    1.0006     1.00099    1.00153    1.00234    1.00421    1.01048    1.25661    3.25602]]
min factor:
[[   0.        10.        20.        30.        40.        50.        60.        70.        80.        90.     ]
 [   0.         1.         1.         1.         1.         1.         1.         1.         1.         0.83243]
 [  15.         1.         1.00003    1.00005    1.0001     1.00013    1.00018    1.00027    1.00046    0.81348]
 [  30.         1.00001    1.00021    1.00042    1.00068    1.00097    1.0014     1.002      1.00329    0.80091]
 [  45.         1.00001    1.00045    1.00093    1.00144    1.00201    1.00311    1.00435    1.0072     0.79633]
 [  60.         1.00001    1.00048    1.00097    1.00154    1.00219    1.00311    1.00471    1.00733    0.79632]
 [  75.         1.         1.0002     1.00041    1.00061    1.00089    1.00122    0.99217    0.93006    0.80115]
 [  90.         0.99823    0.99609    0.99296    0.98919    0.98308    0.97268    0.95135    0.89105    0.81612]
 [ 105.         0.99611    0.9914     0.98589    0.9791     0.97051    0.95727    0.93458    0.88781    0.84123]
 [ 120.         0.99566    0.99025    0.98425    0.97731    0.96865    0.95571    0.93444    0.8944     0.8751 ]
 [ 135.         0.99585    0.99051    0.98483    0.97823    0.97021    0.95887    0.94363    0.92017    0.91384]
 [ 150.         0.99708    0.99322    0.98929    0.9848     0.97942    0.9729     0.96374    0.95287    0.95159]
 [ 165.         0.99873    0.99705    0.99526    0.99329    0.99112    0.98862    0.9852     0.98181    0.9814 ]
 [ 180.         0.99985    0.99964    0.99942    0.9992     0.99895    0.99862    0.99827    0.9979     0.99787]
 [ 195.         0.99874    0.99712    0.99526    0.99332    0.9912     0.98853    0.98504    0.9816     0.9814 ]
 [ 210.         0.99709    0.99317    0.98914    0.98469    0.97923    0.97237    0.96402    0.95294    0.95155]
 [ 225.         0.99584    0.9906     0.98487    0.97819    0.97013    0.95894    0.94337    0.92043    0.91384]
 [ 240.         0.99566    0.99028    0.98429    0.97739    0.96849    0.95569    0.93451    0.89383    0.87507]
 [ 255.         0.9961     0.99128    0.98605    0.97922    0.97018    0.95665    0.93464    0.88813    0.8412 ]
 [ 270.         0.99827    0.99614    0.99306    0.9891     0.98347    0.97246    0.9512     0.89121    0.81606]
 [ 285.         1.         1.0002     1.00037    1.00061    1.00087    1.00134    0.9914     0.92805    0.80112]
 [ 300.         1.00001    1.0005     1.001      1.00157    1.00223    1.00316    1.00469    1.00706    0.79633]
 [ 315.         1.00001    1.00046    1.0009     1.0014     1.00212    1.00293    1.00445    1.00736    0.79632]
 [ 330.         1.00001    1.00022    1.00043    1.00066    1.00103    1.00138    1.00207    1.00334    0.80092]
 [ 345.         1.         1.00003    1.00005    1.00009    1.00013    1.00018    1.00028    1.00047    0.81335]
 [ 360.         1.         1.         1.         1.         1.         1.         1.         1.         0.83229]]
std deviation
[[   0.        10.        20.        30.        40.        50.        60.        70.        80.        90.     ]
 [   0.         0.00005    0.00012    0.00021    0.00032    0.0005     0.0008     0.00159    0.01138    0.58817]
 [  15.         0.00051    0.00122    0.0021     0.00326    0.00493    0.00782    0.01473    0.06154    0.38418]
 [  30.         0.00142    0.00342    0.00585    0.00897    0.01339    0.02068    0.03651    0.099      0.24159]
 [  45.         0.00213    0.00521    0.00883    0.01332    0.01949    0.02915    0.04742    0.09555    0.15017]
 [  60.         0.0021     0.0052     0.00876    0.01302    0.01852    0.02642    0.03917    0.0612     0.1021 ]
 [  75.         0.00128    0.0032     0.0053     0.00759    0.01039    0.01376    0.01777    0.02134    0.09992]
 [  90.         0.00036    0.00098    0.00174    0.00273    0.00421    0.00686    0.01279    0.03274    0.10824]
 [ 105.         0.00108    0.00314    0.00559    0.0086     0.0126     0.01854    0.02911    0.0524     0.10365]
 [ 120.         0.00149    0.00443    0.0078     0.01175    0.01669    0.02349    0.03403    0.05398    0.08351]
 [ 135.         0.00131    0.00399    0.007      0.01048    0.01454    0.01994    0.02785    0.04104    0.05558]
 [ 150.         0.00078    0.00244    0.00427    0.00633    0.00867    0.01174    0.01597    0.02244    0.0282 ]
 [ 165.         0.00026    0.00083    0.00145    0.00214    0.00292    0.00391    0.00523    0.0072     0.00865]
 [ 180.         0.00003    0.00008    0.00014    0.00021    0.00028    0.00038    0.00051    0.00068    0.00081]
 [ 195.         0.00026    0.00083    0.00146    0.00215    0.00292    0.00392    0.00524    0.00717    0.00865]
 [ 210.         0.00078    0.00244    0.00428    0.0063     0.00869    0.01176    0.01597    0.02246    0.02813]
 [ 225.         0.00131    0.004      0.00705    0.01048    0.01459    0.01998    0.0279     0.04099    0.05564]
 [ 240.         0.00149    0.00444    0.00778    0.01175    0.01663    0.02351    0.03409    0.05388    0.08364]
 [ 255.         0.00109    0.00313    0.00559    0.00861    0.01259    0.01855    0.029      0.05254    0.10336]
 [ 270.         0.00036    0.00099    0.00174    0.00274    0.00422    0.00688    0.01283    0.03262    0.10829]
 [ 285.         0.00127    0.00321    0.00532    0.00765    0.01038    0.01377    0.01782    0.02138    0.10016]
 [ 300.         0.0021     0.00521    0.00876    0.01299    0.0185     0.0264     0.03923    0.06137    0.10187]
 [ 315.         0.00215    0.00521    0.00883    0.01336    0.01951    0.02914    0.0475     0.09555    0.15013]
 [ 330.         0.00143    0.00344    0.00586    0.00894    0.01338    0.02072    0.03681    0.09954    0.24286]
 [ 345.         0.00051    0.00123    0.00209    0.00324    0.00492    0.00779    0.01474    0.06034    0.38562]
 [ 360.         0.00005    0.00012    0.00021    0.00033    0.0005     0.00081    0.00157    0.01179    0.59786]]

distance vs bearing, lat in range 0..80:
max factor:
[[     0.           0.01         0.1          1.           3.          10.          30.         100.        1000.       10000.       20000.     ]
 [     0.           1.00333      1.81028      2.68414      2.61964      3.10754      3.25157      3.19962      3.2419       3.15955      1.     ]
 [    15.           1.04825      1.74503      1.92905      2.01209      2.11826      2.12659      2.13008      2.13819      2.12922      1.     ]
 [    30.           1.14428      1.42043      1.4745       1.53725      1.55089      1.56228      1.5638       1.56511      1.56296      1.     ]
 [    45.           1.12441      1.25933      1.26302      1.28975      1.29248      1.29373      1.29278      1.29318      1.29206      1.     ]
 [    60.           1.03106      1.07908      1.12246      1.13287      1.13635      1.1361       1.13742      1.13721      1.12522      1.     ]
 [    75.           1.01977      1.03463      1.04647      1.04586      1.04668      1.04726      1.04685      1.04708      1.           1.     ]
 [    90.           1.00294      1.00332      1.00456      1.00485      1.0051       1.00512      1.00515      1.00512      0.99999      1.     ]
 [   105.           1.           1.           0.99999      0.99994      0.99982      0.99938      0.99922      0.99999      1.           1.     ]
 [   120.           1.           1.           0.99998      0.99985      0.99955      0.99964      0.99999      0.99998      1.           1.     ]
 [   135.           1.           1.           0.99999      0.99986      0.99959      0.99994      0.9999       0.99999      1.           1.     ]
 [   150.           1.           1.           0.99999      0.99994      0.99981      0.99968      0.99997      1.           1.           1.     ]
 [   165.           1.           1.           1.           0.99999      0.99998      0.99998      1.           1.           1.           1.     ]
 [   180.           1.           1.           1.           1.           1.           1.           1.           1.           1.           1.01239]
 [   195.           1.           1.           1.           0.99999      0.99998      0.99996      0.99999      1.           1.           1.     ]
 [   210.           1.           1.           0.99999      0.99994      0.99998      0.9999       0.99995      1.           1.           1.     ]
 [   225.           1.           1.           0.99999      0.99986      0.99959      0.99993      0.99987      0.99999      1.           1.     ]
 [   240.           1.           1.           0.99998      0.99985      0.99975      0.99974      0.99976      0.99998      1.           1.     ]
 [   255.           1.           1.           0.99999      0.99994      0.99982      0.99975      0.99963      0.99999      1.           1.     ]
 [   270.           1.00301      1.00407      1.00509      1.00489      1.00511      1.00512      1.00515      1.00514      1.           1.     ]
 [   285.           1.02476      1.0383       1.04634      1.04468      1.04685      1.0471       1.04713      1.04724      1.           1.     ]
 [   300.           1.08744      1.08419      1.12921      1.13597      1.13683      1.13723      1.13714      1.13749      1.12441      1.     ]
 [   315.           1.13164      1.19722      1.25881      1.28504      1.29028      1.2933       1.29212      1.29346      1.29252      1.     ]
 [   330.           1.02011      1.41233      1.53013      1.52894      1.53543      1.55058      1.56415      1.56542      1.5598       1.     ]
 [   345.           1.00512      1.64577      2.08286      1.99041      2.11933      2.12104      2.12578      2.13419      2.11718      1.     ]
 [   360.           1.00512      1.30189      2.69966      3.0191       2.86451      3.13177      3.16542      3.25602      3.1336       0.99999]]
min factor:
[[     0.           0.01         0.1          1.           3.          10.          30.         100.        1000.       10000.       20000.     ]
 [     0.           1.           1.           0.86728      0.83539      0.83242      0.83297      0.83241      0.83243      0.83183      0.84798]
 [    15.           1.           0.95975      0.83564      0.81706      0.81466      0.81429      0.81406      0.81348      0.81347      0.8436 ]
 [    30.           1.           1.           0.81755      0.80445      0.80352      0.80082      0.80095      0.80091      0.80091      0.84316]
 [    45.           1.           0.80262      0.79752      0.7965       0.79645      0.79646      0.79638      0.79633      0.79639      0.84981]
 [    60.           1.           0.81827      0.79842      0.79725      0.79654      0.79633      0.79636      0.79632      0.79646      0.85816]
 [    75.           1.           0.9928       0.80578      0.8025       0.80181      0.80125      0.80112      0.80115      0.80125      0.87677]
 [    90.           0.85811      0.86776      0.82413      0.81975      0.81693      0.81656      0.81609      0.81612      0.81631      0.8956 ]
 [   105.           0.97607      0.85708      0.84434      0.84383      0.84192      0.84119      0.84124      0.84123      0.84127      0.90661]
 [   120.           0.92794      0.91415      0.88535      0.87684      0.87534      0.87507      0.87506      0.8751       0.87528      0.90824]
 [   135.           0.95918      0.93877      0.91962      0.91522      0.91452      0.91449      0.91389      0.91384      0.91413      0.908  ]
 [   150.           0.98628      0.95765      0.95465      0.95412      0.95179      0.95223      0.95167      0.95159      0.95181      0.90789]
 [   165.           0.99786      0.9901       0.98337      0.98158      0.98146      0.98144      0.98141      0.9814       0.98146      0.90941]
 [   180.           0.99924      0.99862      0.99814      0.99801      0.9979       0.99789      0.99788      0.99787      0.99788      0.90915]
 [   195.           0.9958       0.98343      0.98243      0.98226      0.98156      0.98143      0.98143      0.9814       0.98146      0.91031]
 [   210.           0.97932      0.95723      0.95413      0.95336      0.9518       0.95178      0.95163      0.95155      0.95187      0.90794]
 [   225.           0.96194      0.91501      0.915        0.91408      0.91438      0.91385      0.91405      0.91384      0.91402      0.90719]
 [   240.           0.98537      0.89014      0.87706      0.87617      0.87585      0.87544      0.8752       0.87507      0.87529      0.90694]
 [   255.           0.99164      0.8504       0.84631      0.84763      0.84162      0.84254      0.84152      0.8412       0.84135      0.91007]
 [   270.           0.98552      0.90239      0.82108      0.8217       0.81688      0.81614      0.81616      0.81606      0.81637      0.89812]
 [   285.           1.           0.80738      0.81018      0.80192      0.80142      0.80168      0.80141      0.80112      0.8014       0.87696]
 [   300.           1.           0.82985      0.80037      0.79683      0.79666      0.79633      0.79632      0.79633      0.7964       0.85947]
 [   315.           1.           0.8075       0.81606      0.79858      0.79641      0.79635      0.79632      0.79632      0.79638      0.84933]
 [   330.           1.           0.88805      0.81244      0.80154      0.80118      0.80152      0.80129      0.80092      0.80091      0.84377]
 [   345.           1.           1.           0.81396      0.82175      0.81609      0.81344      0.81344      0.81335      0.81347      0.84325]
 [   360.           1.           0.84768      0.84086      0.83478      0.83405      0.83252      0.83229      0.83229      0.83181      0.8495 ]]
std deviation
[[     0.           0.01         0.1          1.           3.          10.          30.         100.        1000.       10000.       20000.     ]
 [     0.           0.00001      0.00423      0.01955      0.0304       0.06223      0.10945      0.20447      0.58817      0.22426      0.07565]
 [    15.           0.00026      0.00513      0.01061      0.02421      0.04377      0.07536      0.135        0.38418      0.17084      0.07592]
 [    30.           0.00068      0.00376      0.0075       0.01606      0.02806      0.05129      0.09347      0.24159      0.14146      0.07398]
 [    45.           0.00067      0.00256      0.00635      0.01153      0.02124      0.03731      0.06682      0.15017      0.13303      0.06964]
 [    60.           0.00017      0.00171      0.00453      0.00833      0.015        0.02689      0.04671      0.1021       0.12802      0.0632 ]
 [    75.           0.00011      0.00039      0.00365      0.00644      0.01201      0.021        0.03672      0.09992      0.11649      0.05448]
 [    90.           0.00064      0.00071      0.00373      0.00641      0.01205      0.02111      0.03952      0.10824      0.1          0.04523]
 [   105.           0.00016      0.00145      0.0039       0.00753      0.01386      0.02453      0.0428       0.10365      0.07753      0.03538]
 [   120.           0.00036      0.00094      0.00372      0.00666      0.01329      0.02281      0.03992      0.08351      0.0545       0.02593]
 [   135.           0.00032      0.0008       0.00291      0.00551      0.00981      0.01685      0.0293       0.05558      0.03293      0.0177 ]
 [   150.           0.00009      0.00042      0.0015       0.003        0.00544      0.00933      0.01592      0.0282       0.01572      0.01211]
 [   165.           0.00002      0.00014      0.00051      0.00098      0.0018       0.00304      0.00508      0.00865      0.00469      0.00958]
 [   180.           0.           0.00002      0.00005      0.00009      0.00017      0.00029      0.00048      0.00081      0.00043      0.00892]
 [   195.           0.00004      0.00017      0.00054      0.00093      0.00177      0.00301      0.00512      0.00865      0.00467      0.00953]
 [   210.           0.00012      0.00062      0.00151      0.00302      0.00537      0.00941      0.01587      0.02813      0.01579      0.01217]
 [   225.           0.00025      0.00095      0.00283      0.00564      0.00975      0.01696      0.02936      0.05564      0.03302      0.0178 ]
 [   240.           0.00014      0.00113      0.00395      0.00734      0.01304      0.02252      0.03982      0.08364      0.05428      0.02603]
 [   255.           0.00008      0.00141      0.00388      0.00731      0.01439      0.02429      0.04306      0.10336      0.07795      0.03543]
 [   270.           0.00008      0.00061      0.00373      0.00679      0.01246      0.0222       0.03933      0.10829      0.09986      0.04532]
 [   285.           0.00013      0.0013       0.00308      0.00693      0.01169      0.02043      0.03683      0.10016      0.11724      0.05455]
 [   300.           0.00042      0.00151      0.00441      0.00817      0.01525      0.02679      0.04663      0.10187      0.12759      0.063  ]
 [   315.           0.00062      0.00196      0.00618      0.01184      0.02105      0.03746      0.06605      0.15013      0.13312      0.06947]
 [   330.           0.00011      0.00221      0.00962      0.01654      0.02813      0.05005      0.09079      0.24286      0.14154      0.07392]
 [   345.           0.00003      0.00367      0.00989      0.0223       0.04025      0.07489      0.13359      0.38562      0.17313      0.07584]
 [   360.           0.00002      0.00211      0.0239       0.0396       0.06153      0.11233      0.20075      0.59786      0.2286       0.07568]]

The errors behave as one would expect. The greater the latitude, the greater the error. Distances along vertical/horizontal lines have 0 error, the largest errors are along diagonal lines.

For my purposes, I need distances of less than 100km that are below 70 degrees latitude (basically distances within existing cities). For that use case, I can expect errors smaller than 1%.

share|improve this answer
    
you solved it as I did. Still hoping there is some formulaic answer out there. –  Pablitorun Nov 21 '13 at 17:15
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.