Are you triggering an interrupt with the GPIO? If so I think your best option in the ISR handler is to use xTaskNotifyFromISR() to let your task know there's something happening, and xTaskNotifyWait() in the task to wait for the signal.
The time when you hit the ISR is more important than when the task runs, so these functions allow you to pass a 32-bit value, which can be the time offset in whatever scale you want to use. You can do stuff like reserve a bit in the notification value (so use 31-bits for the time) and use that to notify either end if the logging task missed a notify call.
Timing: the standard FreeRTOS clock might not have enough resolution for your purposes, but if it does then the built-in TickType_t has useful functions and is reasonable slim. I though there was a higher-res clock source but I don't have access to the dox at the moment, so might be thinking of a different processor! Either way, you might need to trim the top bits to get everything into the 32-bit notify value. Also note that the task tick is pretty slow on the ESP-IDF, and that might affect how quickly the log task can deal with the ISR notify, although at 40ms between interrupts it should manage.