Don't set 'config.xorg = {}'

This makes memoization of Nixpkgs evaluation less effective, since
some Nixpkgs invocations may have 'config = {}' while others may have
'config = { xorg = {}; }'.

Instead set 'config = {}'.