Custom fatal error handler in SLIM4

I am developing a REST API and want to support only JSON responses. I added a custom error handler this way:

$errorMiddleware = $app->addErrorMiddleware($config['error']['debug'], true, $config['error']['log']['communication']);
$errorMiddleware->setDefaultErrorHandler($container->get(ErrorHandler::class));

It works, but every now and then I got a fatal error, which is rendered by the default HTML renderer and I guess it is not logged as well. Is there a way to handle fatal errors with a custom error handler?

I know from my own frameworks, that this is a tricky question, because if the custom handler raises an error, then the code can collapse into an infinite error loop… But I still want to do this.