Re: Different behavior in library mode (PARI) vs. in GP (when using Z_issmooth)

[Bill Allombert <Bill.Allombert@math.u-bordeaux.fr>]

On Tue, Jun 06, 2023 at 07:52:19PM +0200, Daniel Berger wrote:
> Hello,
> 
> a simplified version of what I am doing is the following code using PARI:
> 
> ```C
> #include <pari/pari.h>
> GEN
> function(ulong B, GEN d, ulong m)
> {
>     GEN I, e, e_, ret = NULL;
>     ulong i;
>     pari_sp ltop = avma;
>     I = const_vecsmall(m+1,1);
>     e = quadunit0(d,-1);
>     if (gsigne(gnorm(e)) == -1) e = gsqr(e);
>     e_ = e
>     for (i = 1; i < lg(I); i++)
>     {
>         if(!gel(I,i)) continue;
>         if(Z_issmooth(gel(e,3),B))
>         {
>             if (ret == NULL) ret =
> mkvec(diviiexact(subii(gel(e,2),gen_1),gen_2));
>             else ret =
> vec_append(ret,diviiexact(subii(gel(e,2),gen_1),gen_2));
>         }
>         else for (j = i; j < lg(I); j += i= gel(I,j) = 0;
>         e = gmul(e,e_);
>     }
>     //copy ret to a clean place in memory
>     return gerepileupto(ltop,ret);
> }

(Your code do not compile for multiple reasons)

You should not use gel() with t_VECSMALL component. use I[1], I[2] etc.

> Now when i run this code as part of compiled C code, it works fine and does
> what it's supposed to do. For example `function(10, 2, 3)` returns `[1, 8,
> 49]`
> 
> If I however compile this as part of a shared library and install it for use
> in gp, it doesn't. It appears to me that `Z_issmooth` always returns 1, for
> example `function(10, 2, 3)` now returns `[1, 8, 49, 288]`. I tried testing
> `Z_issmooth` separately in gp by installing it with
> `install(Z_issmooth,"GU",smoothtest,"libpari.so")` and for example
> `smoothtest(408,10)` (should return 0) but it breaks with a segmentation
> fault.

Z_issmooth returns a 'long' so the code should be "lGU"
? install(Z_issmooth,"lGU",smoothtest)
? smoothtest(408,10)
%2 = 0

> Any insights as to what I might be doing wrong are appreciated.

Send me your correct C file.

Two possibilities come to mind:
- maybe the primetable is not initialised in the same way in the
C program than in GP (Z_issmooth use the primetable).

- maybe your are mixing different copies of libpari.so and get strange result.
Under GP it is safer to omit "libpari.so" in install().

Cheers,
Bill.