I love this guy’s art
Double considered harmful
or: How the number 1.2 cramped my style..
So, everybody who paid attention in college CS knows this. Heck, I didn’t always pay attention, and I knew it, but it still caught me by surprise when I hit my head on this issue in the real world.
Recapping the background: The double (as we know it in Java) is short for “double precision binary floating-point format”, and they are the floats more precise big brother. Floats and doubles are a trade-offs where precision is traded for the ability to store them in a fixed amount of space and to perform very fast arithmetic on them. Basically it’s two seperate numbers, called a significant and an exponent, and they are then subjected to some mathemagic to yield any of a very wide range of numbers.
If you don’t need the floating point, which most people often don’t, you’re better off storing your numbers as fixed-point (e.g. the DECIMAL columntype in most databases). Unfortunately Java (and many other not-so-modern languages) doesn’t have such a type natively.
So doubles aren’t perfectly precise, we know that, but that’s only for large and weird numbers, right? No. For one, a double can’t be 1.2. You wouldn’t know that if you didn’t look for it, because when you print doubles, Java rounds them off for you. And rightly so, the error is exceedingly small: had this been 1.2 meters, the error would amount to a fraction of the size of an atom. Obviously, this is perfectly fine - right up until the point when it’s not.
Our application dabbles quite a bit in arbitrary precision. We (among other things) deal in sequences of daily return ratios and foreign exchange - numbers that are notoriously unwilling to just be nice and round. By the time we get around to adding up your daily FX-adjusted returns for the past year, the error on a simple double-value starts to add up to real money, so we implement these numbers as BigDecimals (specifically our own wrapper for it that adds some useful fuctionality, including the ability to do fractions).
This is all very well - until someone decides to stick something like 1.2 into BigDecimal. Recalling how printing 1.2 is fine, you’d be excused in assuming that using the double-constructor of BigDecimal is fine. But no, and with potentially disastrous consequences.
Consider this simple method:
public BigDecimal iterate_multiplication(
int iterations,
BigDecimal multiplicand) {
BigDecimal bd_out = BigDecimal.ONE;
for (int i=0; i<iterations; i++)
bd_out = bd_out
.add(BigDecimal.ONE)
.multiply(multiplicand);
return bd_out;
}
And these two invocations - on the surface they are equivalent:
iterate_multiplication(1000, new BigDecimal(1.2));
iterate_multiplication(1000, new BigDecimal(“1.2”));
(BigDecimal will parse a decimal number from a string - IMHO the easiest way to get an exact value into a BigDecimal)
The latter invocation returns well over 1000 times faster. Three full orders of magnitude. 15 milliseconds in place of 20 seconds. This is because the former is called with an argument that, behind the scene, has 51 decimal places, which, obviously, is significantly more complicated to multiply.
The two methods returns rather different numbers, but that is due to iterating the multiplication, which amplifies the error of the double. Both numbers have 81 digits before the decimal point, and share the first 13. The really interesting - but at this point, hardly surprising - difference is after the decimal point: The former has 52,000 decimals, the latter just 1,000.
Even though fortune-cookie conclusions are often over-generalizing, I will venture into that territory: Don’t ever construct a BigDecimal with a double. It doesn’t do what you think it does.
——
Written for & posted on the youDevise blog. youDevise is a great place to work for hackers, and we’re hiring!
Beautiful doesn’t make it true
Yesterday, @christianevejlo tweeted interest in whether volcano-grounded planes were a net benefit for the environment. Retweeting an answer, she referenced Information is Beautiful’s post “Planes or Volcano”, an indeed aesthetically pleasing illustration of the CO2 emissions of the European aviation industry next to that of the Icelandic volcano. I replied that “Information is beautiful seems to be more about beautiful than about information..”. She asked me to elaborate, and here goes..
Infographics seems to have had a renaissance recently. There is a lot of information out there, and visualizing it in various ways helps convey this information to a much larger audience. This is a good thing. Information is Beautiful is part of this, and they have some truly stunning work. However, in this case, what was conveyed wasn’t data. It was the point that commercial aviation is evil bad for the environment. The way I can tell is the way they casually observe that Eyjafjallajökull belches out 3000 tons of Sulphur Dioxide (SO2) daily, while they have no interest in this beyond using it as a stepping stone to estimate the amount of Carbon Dioxide (CO2) emitted.
CO2 is a greenhouse gas that may very well have some very negative effects, but they lay decades in the future.
SO2 is what causes acid rain.
Planes emit CO2 - but they emit almost completely clean CO2 (note that bar global warming, CO2 isn’t considered harmful). Volcanoes blast tons and tons of all kinds of nasty and not so nasty stuff into the air. Some of the ash contains nutrients that will fertilize soil, other will, as mentioned, cause acid rain. It’s much more complex than CO2.
Information is Beautiful could have chosen to diagram the amount of SO2 emitted by aviation (next to nothing) next to the amount emitted by the volcano (a lot), but they didn’t - and that’s OK, because it wouldn’t make sense.
The point is that a volcano compared to commercial aviation is apples to oranges. No matter how beautiful the infographics, it’s comparison in itself is pointless.
Now, the general statement that Information is Beautiful is more about beautiful stems from their previous post - “How much do music artists earn online?”. Looking at this, again, very good looking illustration, it’s easy to scroll down and say “boo, evil Spotify”. But until you look at the numbers (which, in infographics, is sort-of the point not to), you don’t realize that it compares apples to oranges.
They equate selling one physical CD with selling one album download with selling one track download with getting one play on a service like Spotify. Or the short version: for this graphic to convey an honest version of the truth, you have to accept the premise that selling a physical CD and getting a play on Spotify is the same.
First, one album download should equate ten, not one, track downloads. And getting a play should acknowledge that a play has zero marginal cost to the user, and that the user may choose to play a track many times. Once you’ve purchased and downloaded a track, you’ve got unlimited plays, forever.
If you accept ten tracks to an album, and ten plays pr. track, the three bottom circles should be 100 times smaller (and the circles relating to single track downloads, 10 times smaller). That would make Rhapsody only twice the low-end retail deal. Considering that Rhapsody, last.fm and Spotify are widely available to artists, and retail deals aren’t, that’s not bad at all. Spotify is still on the cheap end, but only 11 times the low-end retail deal. Considering the enormous long-tail opportunities that the Spotify-like services provide (would you buy, say, Rick Astley’s “Never gonna let you down”? Probably not. Would you play it on Spotify at a party once or twice? That’s more likely), the number 11 shrinks even more.
And what is probably the worst offense: why doesn’t it compare a Spotify play with a radio-play? These services are essentially single-listener-market radio stations.
Again, the agenda is clear. It’s not about the data, it’s about calling Spotify evil. The data may support that conclusion. The graphics doesn’t.
Finally, if you want more, please take a look at the limited edition posters being sold in the right-hand column. Nominally, it’s an illustration comparing the political left with the political right. Read closer. The right is said to be “exclusive”, the left “inclusive”. The left-wing parent is “nurturing” and has a relationship with her child that is build on “respect and trust”, while the right-wing parent is “strict” (as opposed to nurturing) and the relationship is build on “respect and fear”. “Survival of the fittest” is repeated twice on the right side, the left has “One for all and all for one”.
I have no doubt that this is how the urban, american left views itself, I know I’m probably guilty of similar prejudices the other way around. But if it’s an infographic, it’s about american left-wing prejudices, not about political philosophies.
In conclusion: Information is Beautiful makes beautiful things. For information, they are little better than any political blog.
S/SF dropper skattestop, R jubler
»Vi har brudt med skattestoppet. Vi har sagt klart og tydeligt, at der er brug for flere indtægter«, siger Helle Thorning-Schmidt
http://politiken.dk/politik/article926636.ece
I landet med verdens højeste skattetryk er det udtryk for allerhøjeste grad af fantasiløshed når man ikke kan “financiere velfærden” uden at skulle stramme skatteskruen. Mer’ vil ha’ mer’ - hav dog mod til at prioritere!
SSF opgiver skattestoppet! Seriøst gode nyheder sammen med radikale ideer om frihed, vækst og reformer.
http://twitter.com/vestager/status/10667636028
Hahaha. Good one. At Helle lover at hæve skatten er godt nyt for vækst og reformer? Og frihed? Der var en gang de Radikale var de fornuftige, i hvert fald økonomisk, i oppositionen.
Næste gang venstrefløjen kræver flere penge til det offentlige, så husk denne sag. Københavns Kommune har været Socialdemokratisk ledet i 82 år, og ansvaret for indkøbsaftaler ligger på overborgmesterens kontor.
Et ofte hørt argument mod privatisering, er at det vil blive dyrere når der også skal sidde en ejer og tjene på det. Den lader vi lige stå et øjeblik.
