Re: A question about memory allocation in Linux
On Aug 26, 1:46 am, JQ <group...@gmail.com> wrote:
First, thank you all for the suggestions.
Actually, from a preprocessing program, I know the size of the
file and how much memory is needed.
Step 1: I read a small configuration file and allocate the memory by
calling new().
Step 2: I read the data file (the file I mentioned in the initial
post), process the data and write them to the memory. The total mem is
around 800M since it is a large network.
I tried reading the file only, doing simple operations and not
writing them into the mem. It is fast. Seems the memeory
allocation is slow?
If I understand you correctly, you're only doing one memory
allocation, which should be close to negligible. Could you post
the code for the loop where you're reading the data and copying
it into the buffer---even with the most na=EFve implementation,
using std::getline into an std::string, then std::copy into the
buffer, I get reasonable performance on my system.
FWIW: allocating the buffer and reading the entire file into it
in one low-level system request (read under Unix, ReadFile under
Windows) should be very fast, and is fairly simple to do. Under
Unix, mmap isn't that much more complex, and is a little bit
faster; IIRC, memory mapping under Windows is a little bit more
complicated (but that might only be because I'm less familiar
with Windows).
--
James Kanze (GABI Software) email:james.kanze@gmail.com
Conseils en informatique orient=E9e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S=E9mard, 78210 St.-Cyr-l'=C9cole, France, +33 (0)1 30 23 00 34