futex issue on macOS

[radiolistener]

I have limited experience writing code for macOS, but I’ve recently run into a porting challenge.

In my project, I use futex (fast userspace mutex) on Linux for fast lock-free notifications between threads, like this:

Code: [Select]

syscall(SYS_futex, &variable, FUTEX_WAIT, 0, ...);
syscall(SYS_futex, &variable, FUTEX_WAKE, 1, ...);

This allows producer threads to notify a consumer thread to wake up without using any locks, and ensures that no notifications are lost if the consumer thread is not currently waiting.

On Windows, the closest equivalents are auto-reset events (pre-Windows 8 ):

Code: [Select]

_hEvent = CreateEvent(nullptr, FALSE, FALSE, nullptr);
WaitForSingleObject(_hEvent, INFINITE);
SetEvent(_hEvent);

and, starting from Windows 8, functions similar to Linux futex that don’t require explicit initialization:

Code: [Select]

WaitOnAddress(&variable, &variable, sizeof(variable), INFINITE);
WakeByAddressSingle(&variable);

However, while porting this code to macOS, I haven’t been able to find anything that resembles futex. Consequently, on macOS, the only way to send such a notification seems to be acquiring a heavy mutex lock in the producer thread, which can significantly degrade performance. Is it correct?

Does macOS really lack a mechanism for fast lock-free cross-thread notifications inside a process?
How do people usually solve this problem on macOS?

[brucehoult]

Of course pthread_cond works, but is a little heavy.

dispatch_semaphore (in Grand Central Dispatch) is pretty much the standard low-overhead way, and has been around since 2009 in Snow Leopard.

The highest performance would be Apple's private and undocumented __ulock_wait and __ulock_wake. In theory the interface is unstable, but in practice many 3rd party libraries rely on this, as does e.g. Rust.

- Consumer: Atomically check/load the variable, then __ulock_wait(UL_COMPARE_AND_WAIT, &variable, 0, 0); if it matches.

- Producer: Atomically set variable to non-zero if needed, then __ulock_wake(UL_COMPARE_AND_WAIT, &variable, 0);.

An approved way to use _ulock is via Apple's libc++ implementation of std::mutex and std::condition_variable, in which case code will keep working even if Apple later changes the implementation.

[radiolistener]

Of course pthread_cond works, but is a little heavy.

I’d like to clarify one point regarding pthread_cond / std::condition_variable.

These mechanisms inherently require a mutex to be held by the producer when notifying the consumer. Without this lock, they cannot guarantee reliable notification without the risk of losing the event if the consumer has not yet entered the wait state.

In my case, avoiding any locking in the producer thread is critical. The producer must notify the consumer quickly and on-the-fly, without incurring the overhead of context switches or heavy synchronization. That’s exactly why I’m looking for a lock-free notification mechanism on macOS, analogous to Linux futex or Windows WaitOnAddress/WakeByAddressSingle.

The highest performance would be Apple's private and undocumented __ulock_wait and __ulock_wake. In theory the interface is unstable, but in practice many 3rd party libraries rely on this, as does e.g. Rust.

- Consumer: Atomically check/load the variable, then __ulock_wait(UL_COMPARE_AND_WAIT, &variable, 0, 0); if it matches.

- Producer: Atomically set variable to non-zero if needed, then __ulock_wake(UL_COMPARE_AND_WAIT, &variable, 0);.

This looks very promising. Do you know from which macOS versions these functions are available?

An approved way to use _ulock is via Apple's libc++ implementation of std::mutex and std::condition_variable, in which case code will keep working even if Apple later changes the implementation.

Could you clarify what is meant by "via Apple's libc++ implementation of std::mutex and std::condition_variable"? Is this implemented in the compiler as a kludge workaround, or does the system somehow recognize that a mutex used with std::condition_variable should internally use _ulock? It's not entirely clear how this works.

[brucehoult]

The highest performance would be Apple's private and undocumented __ulock_wait and __ulock_wake. In theory the interface is unstable, but in practice many 3rd party libraries rely on this, as does e.g. Rust.

- Consumer: Atomically check/load the variable, then __ulock_wait(UL_COMPARE_AND_WAIT, &variable, 0, 0); if it matches.

- Producer: Atomically set variable to non-zero if needed, then __ulock_wake(UL_COMPARE_AND_WAIT, &variable, 0);.

This looks very promising. Do you know from which macOS versions these functions are available?

10.12

[radiolistener]

10.12

this is bad, I have only 10.11 for testing
Is there any other options for old macOS?

[brucehoult]

this is bad, I have only 10.11 for testing

That is TEN YEARS old, this month.

[radiolistener]

That is TEN YEARS old, this month.

Yeah, I’m not exactly young anymore either - my only test machine is still running macOS 10.11
Installing a newer system just to check if some code compiles isn’t really on my wishlist.
Do you know any service where I could try building and running C++ code on a more recent macOS?

[abeyer]

There are a bunch of providers of Mac VMs (or sometimes just cheap hardware) by the minute for CI/CD runners on Mac, that's probably the easiest if you don't want to upgrade.

[radiolistener]

Using an online macOS service would indeed be very helpful for testing, but unfortunately I don’t have the possibility to pay for such services right now, as I’m living in a war zone.

There are many online C++ compiler services that allow you to compile and run code, but they all seem to run on Linux, which I already have locally. Does anyone know of similar services available for macOS with no need for payment? I don’t actually need to run a server - a simple VM would be fine, even without network access, as long as I can upload code to it and check how it compiles and runs. There is no need to persist data between sessions.