Can I delete the default NS and SOA added by Route 53 to a hosted zone?

Route 53 enforces the requirement that a hosted zone have at least one NS record and one SOA record at any time. If you want AWS to do your DNS hosting, then you really should not be changing these records.