Re: Setting parisizemax when running multiple Pari/GP instances on one system?

[Bill Allombert <Bill.Allombert@math.u-bordeaux.fr>]

On Tue, Jan 09, 2024 at 10:24:17PM +0100, hermann@stamm-wilbrandt.de wrote:
> I determined sum of four squares for Mersenne primes M_2..M_29.
> I used ternary quadratic form method, M_29 took 23:03:02h to compute on AMD
> 7950X CPU.
> 
> Then I switched to using
> https://pari.math.u-bordeaux.fr/Scripts/foursquares.gp
> 
> which took only 200s for M_29, and 315/861/14,198 seconds for
> M_30/M_31/M_32.
> I did computations sequentially on 32GB RAM 7950X CPU with
> hermann@7950x:~$ grep "^parisize" /etc/gprc
> parisizemax = 31G
> parisize = 31G
> hermann@7950x:~$
> 
> https://github.com/Hermann-SW/Mersenne_primes_sum4squares#foursquaresgp
> 
> 
> Next I switched to old Intel Xeon 6126 dual CPU sytem.
> 7950X has single threaded Passmark rating of 4300, while 6126 has 2139
> (half):
> https://www.cpubenchmark.net/cpu_lookup.php?cpu=Intel+Xeon+Gold+6126+%40+2.60GHz&id=3219&cpuCount=2
> 
> But that system has 192GB RAM, and so I configured
> hermann@i6126:~$ grep "^parisize" /etc/gprc
> parisizemax = 188G
> parisize = 188G
> hermann@i6126:~$
> 
> But then I started three instances of GP to compute M_33, M_34 and M_35 in
> parallel.
> After more than an hour I realized that one instance got killed.
> 
> But the other two instances are running fine after nearly 7h:
> 
> top - 22:08:00 up  8:10,  1 user,  load average: 2.00, 2.00, 2.00
> Tasks: 499 total,   3 running, 496 sleeping,   0 stopped,   0 zombie
> %Cpu(s):  4.0 us,  0.1 sy,  0.0 ni, 95.8 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0
> st
> MiB Mem : 192075.6 total,  11274.2 free, 180249.0 used,    552.4 buff/cache
> MiB Swap:   1392.9 total,   1352.7 free,     40.2 used.  10692.8 avail Mem
> 
> PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
> 2243 hermann   20   0  175.1g  87.6g   3544 R 100.0  46.7 402:00.00 gp
> 2261 hermann   20   0  175.1g  87.6g   3736 R 100.0  46.7 401:28.61 gp
> 2890 hermann   20   0    9652   4284   3244 R   0.3   0.0   0:19.43 top
> ...
> 
> My question is, whether running k GP instance on single system should set
> parisizemax to ramsize/k or less?

It depends. Of course you should not use more memory than you actually have,
In general I advise to set it to ramsize/2 to be safe.

parisizemax does not change the amount of memory you need, it just stop
the computation if it require too much memory.

If you start several computations, it is possible one of them use a lot
of memory but not the others, and it is not always obvious to know which
one will need the most memory. As a result, it can be better to use
ramsize/2 for all instances, hoping not all of them need lots of memory
at the same time!

Cheers,
Bill.