Cookies, though inherently not malicious, can be used to track users or store information and are thus the primary culprit for privacy issues.
The IP address is like a home address. Knowing it does not compromise privacy.
An e-mail address itself without a password is just a string of letters or numbers.
A public key, as its name implies, is public. A user posts their public key for transactions or interactions on the internet.