Well I actually would like to run the else block in case an exception did occurred. 

Let me provide an example from my use case which is the management of a database transaction:

    with savepoint(transaction_manager):

        # Let's try to add into the database with some constraints.

        obj = db.add(data)

        db.flush()

    else:

        # Object already in database.

        obj = db.get(data)

With the following context manager:

    class savepoint(object):

        def __init__(self):

            self._sp = None

        def __enter__(self, tm):

            self._sp = tm.savepoint()

        def __exit__(self, exc_ty, exc_val, tb):

            if exc_ty is not None and issubclass(ecx_ty, IntegrityError):

                self._sp.rollback()

                # We have an exception, execute else block. 

                return False

            # All good, we commit our transaction. 

            self._sp.commit()

            return True

I find it quite a pretty, try and fail back way that I can easily replicate in my code without having to prepare and clean up each time with a try/catch.