Parallel Processing of subsets

[Daniel Berger <Daniel.Berger@hs-rm.de>]

Hello,

I want to do computations on subsets of a given set, that is quite large (around 600+ elements). Since there are quite many subsets, I do not necessarily want to process them all, but still as many as possible given my computational resources, using parallel processing.

For smaller sets (~25 elements), i have successfully used parfor, iterating over the number of subsets and using vecextract to get to the subsets themselves.

However, I don't think this approach is optimal for two reasons:

• Each computation only takes a very short amount of time (few ms), so the added overhead wastes resources (or so I assume after having read the resources on using thread in PARI/GP), I think it would be better to split up the data into chunks and let each core compute a bunch before returning and getting a new batch of data.
  
• It would be better for me to start with smaller subsets (like forsubset works) and the above approach doesn't do this.

So my questions are: Is my assumption above correct? What is the easiest and/or best way to do this in PARI/GP?

Kind Regards,

Daniel