I’m working on a Django project where we use Amazon SQS as broker for Celery. When you install the celery[sqs] bundle (currently version 4.1.0) with pip, it will also install the latest version of PycURL which Kombu uses for communicating over HTTP.

On my fresh install of macOS however, starting the Celery worker would raise the following error:

ImportError: The curl client requires the pycurl library.

Which is weird, since there were no errors when I installed celery[sqs]. Digging a little bit deeper in a Python shell reveals that PycURL wasn’t installed correctly:

>>> import pycurl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)

After some searching, it turns out that Apple stopped including OpenSSL headers since OS X 10.11 El Capitan.

To fix this, we first need to install OpenSSL via Homebrew:

$ brew install openssl

Brew warns against symlinking it to /usr/local to prevent issues with the system OpenSSL, so let’s not do that:

This formula is keg-only, which means it was not symlinked into /usr/local,
because Apple has deprecated use of OpenSSL in favor of its own TLS and crypto libraries.

If you need to have this software first in your PATH run:
echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >> ~/.bash_profile

For compilers to find this software you may need to set:
    LDFLAGS:  -L/usr/local/opt/openssl/lib
    CPPFLAGS: -I/usr/local/opt/openssl/include

In order for PycURL to find the OpenSSL headers, we need to tell setup.py which SSL backend to use and where OpenSSL can be found:

$ pip uninstall pycurl
$ pip install --install-option="--with-openssl" --install-option="--openssl-dir=/usr/local/opt/openssl" pycurl

You should have a working PycURL installation on macOS now!

Update: July 5, 2018

Apparently, pip (version 10.0.1) installing the latest version of PycURL (version 7.43.0.2) will immediately fail on macOS High Sierra:

__main__.ConfigurationError: Curl is configured to use SSL, but we have not been able to determine which SSL backend it is using. Please see PycURL documentation for how to specify the SSL backend manually.

Use the following command to install PycURL correctly:

$ PYCURL_SSL_LIBRARY=openssl LDFLAGS="-L/usr/local/opt/openssl/lib" CPPFLAGS="-I/usr/local/opt/openssl/include" pip install --no-cache-dir pycurl

Update: August 8, 2021

If you already have installed OpenSSL earlier via Homebrew, you can find out the paths for LDFLAGS and CPPFLAGS via:

$ brew info openssl