Is this correct? Does my wallet save every single address?
Yes.
The getaccountaddress method will return the same address until it is used (until a payment is received on that address).
But getnewaddress always creates, stores, and returns a new address.