How to Allocate Huge Amount of RAM to Process?
I'll continue today's RAM usage theme... I understand from references around the web that, by default, Linux kernels will only allocate a maximum of 3GB of RAM+swap to a single process. I've been writing a program (in Perl) to perform some numerical calculations on very large data sets, and, by being very careful to do things in the order that minimizes the overlap between different pieces of original and calculated data being needed in RAM+swap, and making sure that, as soon as a given list of numbers is finished with, a block ends, so that Perl's garbage collector gets called, I've got the program to run within this limitation (it's now using just over 2GB at peak.) The trouble is, I've gone about as far as I can with this efficiency drive, and in a couple of weeks, I have to do the same thing, with even bigger data sets. Since I have plenty of physical space (1GB RAM+10GB swap,) I'd like to get the kernel to allocate more to my program. I know there are various patches with names involving "HIGHMEM." My current kernel is 2.6.6, with # CONFIG_NOHIGHMEM is not set CONFIG_HIGHMEM4G=y # CONFIG_HIGHMEM64G is not set CONFIG_HIGHMEM=y Would setting CONFIG_HIGHMEM64G=y and recompiling the kernel help me, or do I need to do something tricksier, please? -- Thanks Dan Hatton <http://www.bib.hatton.btinternet.co.uk/dan/>
Dan Hatton <dan.hatton@btinternet.com> writes:
The trouble is, I've gone about as far as I can with this efficiency drive, and in a couple of weeks, I have to do the same thing, with even bigger data sets. Since I have plenty of physical space (1GB RAM+10GB swap,) I'd like to get the kernel to allocate more to my program. I know there are various patches with names involving "HIGHMEM." My current kernel is 2.6.6, with
# CONFIG_NOHIGHMEM is not set CONFIG_HIGHMEM4G=y # CONFIG_HIGHMEM64G is not set CONFIG_HIGHMEM=y
Would setting CONFIG_HIGHMEM64G=y and recompiling the kernel help me, or do I need to do something tricksier, please?
By my reading CONFIG_HIGHMEM64G is to support systems with lots of physical RAM, not to expand the process address space (which in any case go beyond 4GB on a 32-bit platform). Time to buy an amd64 system, or figure out a less address-space-intensive way to do your block transfer computations. -- http://www.greenend.org.uk/rjk/
On Thursday 29 July 2004 09:47, Richard Kettlewell wrote:
Time to buy an amd64 system, or figure out a less address-space-intensive way to do your block transfer computations.
But if you do that be aware that none of the socket 754 boards I have seen so far seem to support more than 3GB of physical ram. 939 does a bit better but to get over 4GB you would have to move to a opteron, Not going to be cheap. Maybe the 3GB limit has more to do with the maximum size of module readily available, but that would still be a limitation.
participants (3)
-
Dan Hatton -
Richard Kettlewell -
Wayne Stallwood