Do not remove the screw completely; just loosen until it freely slides in its bearing.

Screw in handle base turns counter clockwise to remove