Adventures in JVM Serialization - part 2

So I got the software from 4 feet to run, though I wish he included all the files so that I could just import the project rather than have to hack odds and ends together and create a new manifest.

I'm a bit disappointed, I would have liked a bigger data set, but this is a starting place.  On my Nexus One, the JSON code took from 410-156ms, with the higher numbers at the beginning of the run, the typical being about 178.  With Protocol Buffers 2.2.0, compiled in, we get from 210-116, typically about 122.  With Protocol Buffers 2.3.0, using the lite JAR, we get from 245-93, 117 typical.

The code is not quite androids vs. androids, so I'm not entirely happy, but it's enough to confirm the other findings, so I guess I"m happy enough to call it a night.

JVM Serializers for Android - Part 1

I've been thinking a lot about Android <==> Server communications lately.  Well actually for a long time now.  The issues that I care about, of course are:

1. Message Size
2. Size of Library
3. Memory usage
4. Overall speed
5. Compression Speed
6. Decompression speed
7. Ease of Integration

I started thinking I was going to use XML, but I hate XML. Then I thought I'd use some kind of simple JSON, and I found the JSON Schema which actually has a draft RFC and tools.  I actually wrote quite a bit of code around JSON. Then I remembered Google's Protocol Buffers, which has versioning as a feature, and produces very small messages.

Fortunatly, I'm not alone on my quest, I found Serializing Data - JSON vs. Protocol Buffers, I was planning to just rerun his tests on Froyo, review his code, and be done with it.  (Froyo has a different 'faster' Dalvik VM, and is suposed to have much better JSON parsing.) But no, I found JVM-Serializer Benchmarks,  it has lots of great data, but is still missing one of my key data points, how much memory does it use.  The benchmarks have unfortunately pointed me to two additional projects Kryo and ProtoStuff.

Kryo seems to be the overall winner, but it doesn't really support versioning.  That's probably ok for my needs, but I need to really think about that as there may be somewhere an issue.  The other question is with the faster JSON implementation, and it being in the ROM, do I wish to pay the cost for an additional library.

It's now time for the beach, so I'll write more on this tonight or tomorrow.  For that I'll have modified the android test program to add in Kryo, and possibly ProtoStuff.

java.lang.VerifyError on Android

For the, you learn something everyday file.

Google shows you in the Backwards Compatibility and Using the Contacts API articles the right ways to write code for two versions of an API, but they don't tell you why this is important.

It turns out that JVM and Dalvik do a byte code verify on class load, and if there's something that's not supposed to be there, like a new API call, it won't load the class. (Ya, well duh!) I've read about it over the years, but never needed to do it until recently. It's of course fairly easy to fix, once you realize it, you've just got to put your new API code in it's own class, and only load the class when you need to. Or you can do it via reflection. I.e. it's a pain, but useful if you want just one binary. (APK, JAR, WAR, etc.)

This time is Different - 800 years of research

One always hopes that Empiricism and Theory will match up. Otherwise, what's the point? We've currently got a lot of economists and politicians who are saying we must be austere. Well, as Paul Krugman points out, what's the evidence for that?

Economists Who Did Their Homework (800 Years of It) - NYTimes.com: "Like a pair of financial sleuths, Ms. Reinhart and her collaborator from Harvard, Kenneth S. Rogoff, have spent years investigating wreckage scattered across documents from nearly a millennium of economic crises and collapses. They have wandered the basements of rare-book libraries, riffled through monks’ yellowed journals and begged central banks worldwide for centuries-old debt records. And they have manually entered their findings, digit by digit, into one of the biggest spreadsheets you’ve ever seen.

Their handiwork is contained in their recent best seller, “This Time Is Different,” a quantitative reconstruction of hundreds of historical episodes in which perfectly smart people made perfectly disastrous decisions. It is a panoramic opus, both geographically and temporally, covering crises from 66 countries over the last 800 years."

History runs in cycles, as much as we want to deny it. Want to learn those cycles, then look at a real data set, not just 30-50 years.

I'm looking forward to reading the book.




I think Rogoff/Reinhart's paper Growth in a Time of Debt is worth reding.

We study economic growth and inflation at different levels of government and external debt. Our analysis is based on new data on forty-four countries spanning about two hundred years. The dataset incorporates over 3,700 annual observations covering a wide range of political systems, institutions, exchange rate arrangements, and historic circumstances. Our main findings are: First, the relationship between government debt and real GDP growth is weak for debt/GDP ratios below a threshold of 90 percent of GDP. Above 90 percent, median growth rates fall by one percent, and average growth falls considerably more. We find that the threshold for public debt is similar in advanced and emerging economies. Second, emerging markets face lower thresholds for external debt (public and private)—which is usually denominated in a foreign currency. When external debt reaches 60 percent of GDP, annual growth declines by about two percent; for higher levels, growth rates are roughly cut in half. Third, there is no apparent contemporaneous link between inflation and public debt levels for the advanced countries as a group (some countries, such as the United States, have experienced higher inflation when debt/GDP is high.) The story is entirely different for emerging markets, where inflation rises sharply as debt increases.

More of Ken Rogoff's Writings