In Objective-C what method keeps long-running parallel tasks running with high CPU resources? (GCD concurrent queue drops priority after 40 seconds) [closed]

BACKGROUND: (skippable?) For 6+ months I've been running code 24/7 to do certain large math calculations -- the specifics aren't important, but it involves 60,000+ digit long integers with the need to do a combination of addition, subtraction, multiplication, division, modular exponentiation, and GCD. For most of that time I have used SageMath on MacOS (where SageMath uses Jupyter to run in Safari) and written the scripts in Python for SageMath. My Mac has 10 CPU cores, so I typically start 8 tabs in SageMath/Jupyter to run 8 instances of the scripts simultaneously, each of which get run on a different core (presumably) and show up as 8 Python processes in the Mac's Activity Monitor.app each utilizing nearly 100% CPU (near full utilization of 1 core). Recently, I have been exploring approaches that might be more efficient. And I coded one of the calculations in C / Objective-C using GMP for the large-integer manipulations, and it was about 12% faster than the equivalent operation in SageMath (about 190 seconds rather than 220 seconds in SageMath). So then I set out to re-write everything, all in Objective-C and GMP in a way that would keep 8 calculations running in parallel (for a similar result to what I was achieving with 8 separate tabs / processes via SageMath) but gain that 12% efficiency improvement. But I am running into difficulty with the macOS cutting off CPU resources (downgrading priority?) for my long-running tasks in the Objective-C app after a brief initial period where it give me 800% CPU utilization (8 cores). CODE SUMMARY: I created an Objective-C app that eventually (after various UI to accept configuration parameters, etc) creates a GCD concurrent dispatch queue. self.concurrentQueue = dispatch_queue_create("com.me.MyAppConcurrentQueue", DISPATCH_QUEUE_CONCURRENT); And it creates a semaphore to manage running only upto 8 calculation tasks in parallel at once. #define MAXIMUM_CONCURRENT_OPERATIONS 8 //... self.maxOperationsSemaphore = dispatch_semaphore_create(MAXIMUM_CONCURRENT_OPERATIONS); The semaphore is used to prevent more than 8 calculation blocks from dispatching, but when they ultimately dispatch a block it looks like this. if (self.calculationBlock1) { dispatch_async(self.concurrentQueue, ^{ self.calculationBlock1(currentIndex, nDict); }); self.blocksDispatched += 1; } At the end of the calculationBlock1 it signals the semaphore (to allow another calculationBlock1 to run) and dispatches a completion block back to the main queue / main thread. dispatch_semaphore_signal(weakSelf.maxOperationsSemaphore); if (weakSelf.completionBlock) { dispatch_async(dispatch_get_main_queue(), ^{ weakSelf.completionBlock(index, results); }); } The completionBlock handles storing my results, dispatching a new calculationBlock1 to the concurrentQueue, etc. All of that all works. There are no problem in terms of the functionality being incorrect, or logic bugs, etc. The program keeps 8 of my big-int calculations running in parallel. THE PROBLEM: When everything starts executing it looks great. The Mac's Activity Monitor.app shows my app using just under 800% CPU (fully utilizing 8 cores, one for each calculation). Remember that each of these calculations takes about 220 seconds in SageMath, and in an early test (performing 10 of them in serial) they took about 190 seconds each in Objective-C using GMP. Well, after about 45 seconds it seems the MacOS cuts the priority of my tasks, or something. The Mac's Activity Monitor.app shows my app's CPU % dropping from 800% to typically 120% - 160%. Remember, thats for all 8 tasks running together, so the end result is they end up taking much longer time to finish executing. Note, my Mac is otherwise sitting idle -- there is no other workload interrupting the task. The Mac just decides to stop devoting CPU time to the calculation blocks in my concurrent queue. FIRST ATTEMPTED FIX, & AN INTERESTING OBSERVATION REGARDING SOURCE OF THE PROBLEM: Googling various descriptions of my problem did not yield great results. I got a lot of boilerplate type suggestions about looking for resource contention, global lock contention, changing the granularity of the work dispatched, checking for memory bottlenecks, etc. The advice was solid, but none of the suggestions I came across seemingly had anything to do with my problem. One suggestion I found (I think when asking Google Gemini for suggestions) was to try creating the concurrent queue differently. Specifically, it suggested adding a flag for the call of the quality of service to provide for the tasks in the queue. This sounded very promising! self.concurrentQueue = dispatch_queue_create("com.me.MyAppConcurrentQueue", dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_CONCURRENT, QOS_CLASS_USER_INITIATED, 0)); Unfortunately, it seemed the same problem existed. After around 45 seconds or so my app dropped from using 800% CPU to somewhere around 120-160%. It just seemed like the GC

May 15, 2025 - 16:00

BACKGROUND: (skippable?)
For 6+ months I've been running code 24/7 to do certain large math calculations -- the specifics aren't important, but it involves 60,000+ digit long integers with the need to do a combination of addition, subtraction, multiplication, division, modular exponentiation, and GCD. For most of that time I have used SageMath on MacOS (where SageMath uses Jupyter to run in Safari) and written the scripts in Python for SageMath. My Mac has 10 CPU cores, so I typically start 8 tabs in SageMath/Jupyter to run 8 instances of the scripts simultaneously, each of which get run on a different core (presumably) and show up as 8 Python processes in the Mac's Activity Monitor.app each utilizing nearly 100% CPU (near full utilization of 1 core). Recently, I have been exploring approaches that might be more efficient. And I coded one of the calculations in C / Objective-C using GMP for the large-integer manipulations, and it was about 12% faster than the equivalent operation in SageMath (about 190 seconds rather than 220 seconds in SageMath). So then I set out to re-write everything, all in Objective-C and GMP in a way that would keep 8 calculations running in parallel (for a similar result to what I was achieving with 8 separate tabs / processes via SageMath) but gain that 12% efficiency improvement. But I am running into difficulty with the macOS cutting off CPU resources (downgrading priority?) for my long-running tasks in the Objective-C app after a brief initial period where it give me 800% CPU utilization (8 cores).

CODE SUMMARY:
I created an Objective-C app that eventually (after various UI to accept configuration parameters, etc) creates a GCD concurrent dispatch queue.

self.concurrentQueue = dispatch_queue_create("com.me.MyAppConcurrentQueue", DISPATCH_QUEUE_CONCURRENT);

And it creates a semaphore to manage running only upto 8 calculation tasks in parallel at once.

#define MAXIMUM_CONCURRENT_OPERATIONS 8
//...
self.maxOperationsSemaphore = dispatch_semaphore_create(MAXIMUM_CONCURRENT_OPERATIONS);

The semaphore is used to prevent more than 8 calculation blocks from dispatching, but when they ultimately dispatch a block it looks like this.

if (self.calculationBlock1)
{
    dispatch_async(self.concurrentQueue, ^{
        self.calculationBlock1(currentIndex, nDict);
    });
    self.blocksDispatched += 1;
}

At the end of the calculationBlock1 it signals the semaphore (to allow another calculationBlock1 to run) and dispatches a completion block back to the main queue / main thread.

dispatch_semaphore_signal(weakSelf.maxOperationsSemaphore);
if (weakSelf.completionBlock)
{
    dispatch_async(dispatch_get_main_queue(), ^{
        weakSelf.completionBlock(index, results);
    });
}

The completionBlock handles storing my results, dispatching a new calculationBlock1 to the concurrentQueue, etc. All of that all works. There are no problem in terms of the functionality being incorrect, or logic bugs, etc. The program keeps 8 of my big-int calculations running in parallel.

THE PROBLEM:
When everything starts executing it looks great. The Mac's Activity Monitor.app shows my app using just under 800% CPU (fully utilizing 8 cores, one for each calculation). Remember that each of these calculations takes about 220 seconds in SageMath, and in an early test (performing 10 of them in serial) they took about 190 seconds each in Objective-C using GMP. Well, after about 45 seconds it seems the MacOS cuts the priority of my tasks, or something. The Mac's Activity Monitor.app shows my app's CPU % dropping from 800% to typically 120% - 160%. Remember, thats for all 8 tasks running together, so the end result is they end up taking much longer time to finish executing. Note, my Mac is otherwise sitting idle -- there is no other workload interrupting the task. The Mac just decides to stop devoting CPU time to the calculation blocks in my concurrent queue.

FIRST ATTEMPTED FIX, & AN INTERESTING OBSERVATION REGARDING SOURCE OF THE PROBLEM:
Googling various descriptions of my problem did not yield great results. I got a lot of boilerplate type suggestions about looking for resource contention, global lock contention, changing the granularity of the work dispatched, checking for memory bottlenecks, etc. The advice was solid, but none of the suggestions I came across seemingly had anything to do with my problem. One suggestion I found (I think when asking Google Gemini for suggestions) was to try creating the concurrent queue differently. Specifically, it suggested adding a flag for the call of the quality of service to provide for the tasks in the queue. This sounded very promising!

self.concurrentQueue = dispatch_queue_create("com.me.MyAppConcurrentQueue", dispatch_queue_attr_make_with_qos_class(DISPATCH_QUEUE_CONCURRENT, QOS_CLASS_USER_INITIATED, 0));

Unfortunately, it seemed the same problem existed. After around 45 seconds or so my app dropped from using 800% CPU to somewhere around 120-160%. It just seemed like the GCD APIs didn't want to continue to devote full CPU resources to long-running tasks. However, with a bit of further testing I noticed something interesting. If I clicked on the GUI of my app, then in Activity Monitor it would show the CPU usage jump back up to 800% for another 45 seconds again before ultimately dropping off again. So it wasn't just an initial 45 seconds of full high-priority access to the CPU for the tasks in my concurrent queue, but a 45 second chunk of full high-priority access to the CPU every time the user interacts with the GUI of my app (just clicks on a window or moves a window). And then it always ultimately drops back down to the lower-priority only giving the app 120-160% CPU even when the Mac is 80% idle (no other apps running, just minimal ~5% utilization by the OS).

SECOND ATTEMPTED FIX, AN UNSUSTAINABLE HACK THAT SHOCKINGLY WORKS:
I know this is a totally unsustainable hack. But because clicking on the GUI of the app seemed to cause the OS to devote more CPU resources to the tasks in the concurrent queue, at least for a brief 45 second window, I decided to just have a second program (an AppleScript) click on the GUI of my first app every 40 seconds. It had over a decade since I had written AppleScript, so I asked Gemini to generate it for me. And then had to sort out some permissions to get it to run properly. (I guess this has changed in the MacOS in the last 10+ years since I wrote AppleScript.) Anyway, so then I had my first program described above running, and a second AppleScript running that just clicked on the GUI of the first app every 40 seconds. And surprisingly, that "fixed" the problem. By that I mean, the OS kept devoting a full 800% of the CPU (8 cores) to the app giving it a full core for each task in my concurrent queue. So this horrible hack allowed me to achieve what I set out to -- re-implementing my big-int calculations running 8 in parallel in a single Objective-C app using 1 core per calculation like I did with SageMath, but doing so about 12% faster than SageMath. Unfortunately, this really isn't a workable usable solution. Mainly because it makes using the Mac for anything else nearly impossible. (CPU isn't the problem, that't why I only do 8 tasks at a time not 10.) The problem with the AppleScript hack is that every 40 seconds the foreground app on the system gets changed to the calculation app. So that's 90 times an hour. Try writing an e-mail or surfing the web, and having the foreground app swap 90 times an hour when you didn't want to and you'll go absolutely mad. I'm sure there has to be a better solution out there!

QUESTION / SOLUTION SOUGHT:
As the title says, what is the best approach in Objective-C to keep long-running parallel tasks running such that the OS continues to provide them the maximum CPU resources? Clearly the right solution doesn't involve a second program to click your GUI. LOL. Do I need to switch to some other parallel-programming approach other than using Grand Central Dispatch (GCD)? (Hopefully not, but if that's the answer so-be-it.)