Re: emscripten and mmap()

[Bill Allombert <Bill.Allombert@math.u-bordeaux.fr>]

On Mon, Apr 03, 2017 at 01:58:24PM +0200, Jeroen Demeyer wrote:
> Moving discussion from bug #1912 to pari-dev
> 
> >This patch causes the mmap code to be more fragile and to break in a
> >non-obvious way on systems where it used to work.
> 
> With "this patch" I assume that you mean my collection of commits involving
> mmap() with PROT_NONE 

Specifically f7d82845952ec92a5c3fa6f1b8b42236f9d80c21

> and that "systems where it used to work" refers to
> emscripten?

... and cygwin at least (that is what #1912 is about).

> As I tried to say several times already, emscripten needs to *emulate*
> mmap(). It has no access to the operating system, so it cannot do an actual
> mmap() system call. 

But it has access to data structures that the browser can implement using
mmap.

> Apparently, the emulation of mmap() in emscripten lacks
> features: for example, MAP_FIXED and the PROT_... flags are simply not
> supported. 

Do you have a reference for that ?

I have made various experiments and it is not as clear cut.
As long as parisizemax is below 128M, then it works fine, but
above that then it crashes more and more easily.

One of my tests (run using node.js) looks like this:
default(parisizemax,2^28);
n=5000
sizedigit(trace(matdiagonal(vector(n,i,i!))))

with the old mmap code it works correctly for all values of n and
parisizemax I tried.
With the new one, it works with (n=3000,parisizemax=2^28) and
(n=4000,parisizemax=2^27) but not with (n=4000,parisizemax=2^28) and
(n=5000,parisizemax=2^28).

The error is something like:
Cannot enlarge memory arrays. Either (1) compile with  -s TOTAL_MEMORY=X
with X higher than the
current value 2039988224, (2) compile with  -s ALLOW_MEMORY_GROWTH=1
which adjusts the size at runtime but prevents some optimizations, (3)
set Module.TOTAL_MEMORY to a higher value before the
program runs, or if you want malloc to return NULL (0) instead of this
abort, compile with  -s ABORTING_MALLOC=0

The value 2039988224 is a bit suspicious.
(I build with -s ALLOW_MEMORY_GROWTH=1).
Note that emscripten is 32bit only, maybe there is an integer overflow
somewhere.

On the other hand, if instead I do only
default(parisizemax,2^30);
the old version works but the new version fails with
Cannot enlarge memory arrays. Either (1) compile with  -s TOTAL_MEMORY=X
with X higher than the
current value 1342177280, (2) compile with  -s ALLOW_MEMORY_GROWTH=1
which adjusts the size at runtime but prevents some optimizations, (3)
set Module.TOTAL_MEMORY to a higher value before the
program runs, or if you want malloc to return NULL (0) instead of this
abort, compile with  -s ABORTING_MALLOC=0

> It would be easy to add a configure check to check whether mmap(MAP_FIXED,
> ...) actually works. It seems that config/has_mmap.c is only *compiled*
> though, not *run*. Testing for MAP_FIXED support can only be done by
> *running* code.

The configure check is as you set it in your patch. It is true it is not
executed, but alas, executing it does not show any problems.

Cheers,
Bill.