Re: Working on iOS port

[ProtonMail <mshm@protonmail.com>]

> On Dec 18, 2017, at 11:39 AM, Bill Allombert <Bill.Allombert@math.u-bordeaux.fr> wrote:
> 
> On Sun, Dec 17, 2017 at 08:15:46PM -0500, ProtonMail wrote:
>> Hi all,
>> 
>> I’m Sharif!  I’m new to the mailing list and pari-dev scene but I’ve
>> used Pari/GP before in my undergraduate research.  I’ve been working
>> for the past two weeks to port Pari/GP to iOS (iPhone, iPod, and
>> iPad).  What I’ve found is that it's really hard compiling Pari/GP for
>> iOS, especially since I don’t have that much experience working with
>> large C projects nor Xcode build tools.
> 
> Hi Sharif!
> 
> Thanks for reporting about this!
> 
> There is quite old link about this topic:
> <http://sagemath.blogspot.fr/2010/06/pari-on-ipad-component-of-sage.html>
> 

I’ve actually seen that link before in the Google rabbit hole I went down when I was trying to compile pari for iOS.  I haven’t gotten around to trying the method outlined in that article yet, but I plan on doing it.  What confuses me though is that you’ll end up with an executable after building pari with the custom build script at the top of the article.  If my assumption is correct, how would you get that executable on the device without jailbreaking it to gain access to the file system?  Is it possible to bundle the executable with an iOS app so that the app can interface with the executable?  I know that it is possible to include a static library on iOS, but I was having trouble compiling libpari.a for arm64.  I wasn’t aware of the cross-compiler though, so I’ll use the info from that webpage as a starting point to produce an arm64 version of libpari.a and go from there.

>> I’ve built apps before and
>> have written a good amount of C, but nothing like this.  So I decided
>> to go the Emscripten route.  I built pari using the Emscripten build
>> commands found at
>> https://pari.math.u-bordeaux.fr/archives/pari-dev-1610/msg00015.html
>> and ran it on iOS using JavaScriptCore, iOS’s built-in JavaScript
>> engine.  It works surprisingly well.
> 
> Indeed! Do you know whether JavaScriptCore support WebAssembly ?

It does!  But only on iOS 11 and up, according to https://caniuse.com/#feat=wasm.

> 
> How does your app compares to
> <https://pari.math.u-bordeaux.fr/gp.html> ?

Unfortunately, that webpage crashes on my iPhone using Safari and Firefox.  I suspect that it’s trying to allocate too much memory.  One curious thing that I’ve found with running Pari in my app is that it immediately claims the max stack size in memory when launching.  For example, if I set the initial stack size to 4MB and the max stack size to 1GB, JavaScriptCore’s memory footprint will immediately balloon to 1GB upon running gp_embedded_init.  I don’t know if it’s Emscripten’s or Pari’s doing or a strange interaction between Pari and Emscripten.  Although I can confirm that this happens when visiting that webpage with Safari and Firefox on my Mac.  I think if you were to set the maximum stack size to something smaller it would alleviate the problem and make it accessible on mobile.

Performance-wise, running GP in JavaScriptCore on my iPhone and even my iPad is much slower than using the browser GP implementation on my computer.  Running fibonacci(1000000) on my iPad took between 3 to 5 minutes, whereas that same code running on my computer ran instantaneously.  Since both times I used the javascript GP interpreter, I think the bottleneck is the mobile processor and not the JavaScript.  That makes me think that, while compiling Pari natively for iOS is definitely nice and the “correct” and most robust way to port Pari to iOS, running Pari in JavaScriptCore may be good enough, at least until I or someone else can compile it natively.  JavaScript engines have gotten remarkably fast recently, and since JavaScriptCore supports WebAssembly, we may be approaching compiled code speeds.

Feature-wise, it does a little more than the browser GP implementation.  Most notably, my app exposes iOS’s file system to Pari through some code that bridges Emscripten and NSFileManager.  You can create and edit GP scripts, save them, and load them into GP at a later time.  write(), read(), readstr(), etc also work since they go through Emscripten, which itself uses my bridge code.  Other than that though, it is for all intents and purposes identical to GP on the website.

> 
>> JavaScriptCore allows data to
>> flow into and out of the JS virtual machine, which allows me to “hide”
>> the JavaScript stuff behind a native UI and native functions (like
>> file system operations).  I’ve essentially bridged a text area to
>> Emscripten’s stdout/stderr functions and a text input to gp_embedded.
>> Here are the features directly related to Pari/GP that I’ve developed
>> so far:
> 
> How do you deal with the asynchronous nature of javascript ?

Well, I use only the synchronous parts of JavaScript :).  It turns out that only a few (important) parts of JavaScript are truly asynchronous, including XHR/Ajax and access to IndexedDB, both of which I avoid.  I just point JavaScriptCore to the pari javascript file bundled with the app, so I don’t need XHR.  That’s also the reason why I opted to bundle the memory file with emscripten-compiled pari file, so I won’t need XHR to load the memory file.  And I don’t use IndexedDB, so that’s not a problem.  Additionally, the javascript code is not running in a browser, so there is no need to wait for the DOM to be ready and no need for handling any events like button clicks and form submissions.  The Emscripten-compiled code is loaded into JavaScriptCore with a simple function call and I proceed to use it by calling gp_embedded through Module.ccall.  All of this is synchronous (or I should say, done on a single background thread to prevent the UI from locking up).

> 
>> P.S. I built Pari/GP with a slightly modified build command than the
>> one in the original post linked above:
>> 
>> make -C Oemscripten-javascript "CC_FLAVOR= -s ALLOW_MEMORY_GROWTH=1 -s PRECISE_I64_MATH=1 -s EXPORTED_FUNCTIONS=[\'_main\',\'_gp_embedded\',\'_gp_embedded_init\'] --memory-init-file 0”
> 
> I found that -s ALLOW_MEMORY_GROWTH=1 was slowing done GP too much, so now
> I use -s TOTAL_MEMORY=1073741824 which limits the memory to 1GB.
> WebAssembly seems not to have this problem.

Thanks, I’ll try that next time!  I might have to reduce the memory limit to 0.25GB or lower because the iPhone 6 and below only have 1GB of RAM.

> 
> How do you handle the optional packages ? (galdata, elldata, etc.)

So far I’ve only been working with the core Pari/GP functionality.  However, now that I’m looking at the optional packages, they all appear to be data files.  If that’s the case, then I can follow the instructions under the section that explains how to install them with an existing pari installation.  Not only does Emscripten have an in-memory file system, it creates several directories that you would expect on a standard linux computer.  When I run default on pari on my iPhone, it says that the datadir is /usr/local/share/pari, so I could put a menu somewhere in the app in which users could choose which optional packages to load into pari, at which point I’d load them into Emscripten’s file system in the datadir.  I could even do this after Emscripten is set up but before pari initializes so pari would have those packages available to it from the start.  I haven’t tried any of this yet, but I can’t see why it wouldn’t work.  Emscripten and Pari are quite resilient :).  The only issue that I can see is that the data would be too large to fit in memory, but the largest package is seadata-big at 220MB uncompressed, which is not unmanageable.  Things start getting dicey when the app uses more than 0.75GB of memory, so limiting the stack size to 0.25GB should give us enough room to load every single package and stay below or not far beyond that threshold.  We could also use the smaller versions of the packages.

Regards,
Muhammad-Sharif Moustafa

P.S. I’m wondering why my name shows up as “ProtonMail” in the email thread and not my actual name.  Do I have to register with the mailing list or is ProtonMail just not good at dealing with mailing lists?