boost: Clean, reducing duplication

This was motivated originally by my cross work, but that goal requires a
few more commits to other things. Still, it's good to start the cleanup
now / get things out of the way.