As I understand it, the problem that both Nix and Docker try to solve is “How do I bundle and run this application in such a way that its dependencies are explicitly specified and don’t interfere with anything installed on the host system”.
They have different approaches, but I think that goal is the same?
Unless it has changed recently, Docker is not intended to be a security layer as far as I know.