This reverts commit d34f5b65. PIL and NumPy don't like this change and I'd rather spend time on bringing wheels to Nix than fixing old infrastructure.