Re: can Inflater be used to uncompress GZIP data?
chattycow wrote:
The data is comming from a socket/network inputstream delivered to me
~32K at a time.
No doubt pipedinput/output streams are an option. However, there is a
few limitations with pipeinput/output streams...namely the static 1K
buffer, the need to connect them, 2 classes as apposed to 1, and any
other overhead associated with these...I'm sure there are more though.
I don't really see any limitations there. The 1K is not really a
limitation. The thread pushing data will have to block at some point.
Whether it blocks at 1K or 32K is not going to have any real impact that
I can see.
The fact that it is 2 classes makes sense because you have to
abstractions used from 2 different threads. Each end of the pipe gets
its own object.
What I wrote is a very simply extension of inputstream and outputstream
(~300 lines..w/ comments) connected by a single bytebuffer array.
Which is not as good of a design as putting it into 2 separate classes.
I write blocks(32K at a time) to the output stream (that I extended).
In another thread, I connect the GZIPInputStream to the inputstream(I
extended) and start to uncompress.
But I'm really not trying to push you to the piped stream or your
implementation. I think both are the wrong idea. What you have is a
pull-push-pull design. You have one thread that is pulling data from the
server then pushing it to a buffer and you have another thread that is
reading from the GzipInputStream which is pulling from the buffer. I see
no reason to do all that. You should be able to that with just a pull
system where the GzipInputStream pulls from the server and eliminate the
middleman. But you have not been forthcoming on how you get the data
from the server to be able to advise you more completely.
SequenceInputStream is fine if you already have all your streams
identified, but I don't have all the data yet and I can't put it all in
memory. SequenceInputStream will combine all the streams that you tell
it, but it can't combine streams on the fly as you add them so it
doesn't help.
As someone else pointed out you can set up your own enumeration so that
is not true.
There are two features I'm trying to get:
1) The socket connection is also being used for general communication
before and after the gzip file is transfered to transmit other items,
informational messages, etc.. If I give the socket stream over to
GZIPInputStream, when it completes it will close the entire connection.
In addition, GZIPInputStream doesn't know it's done until it reads a
"-1" from the inputstream...it assumes it's reading from a file
connected to an input stream.
2) I want to be able to give a status of the file download while it's
downloading via another monitoring thread. As far as I know, you can't
do that with a stream that your already using for something else....so
every 32K, I check to make sure the data is OK and count how much I've
sent/received, how fast, etc.
I'm sure everything I just said brought up a lot more questions? I
posted back because I wanted others to see what my solution was so that
they might have an answer when they come accross the same problem.
I still see nothing that justifies creating the other thread and using
the push-pull-push design. The simplest thing to do here is to create
your own implementation of InputStream that wraps your server
connection. That implementation would contain the logic that is
currently in your thread that is pulling the data from the server it's
just that it won't be done in a loop, but instead will respond to calls
for more data.
If you were to post the code that is reading from the server and pumping
it to your new class then prehaps I can show you how it would look.
In any case, the problem is solved.
Solved quite incorrectly in my opinion.
--
Dale King