This question comes up regularly in the community. A link to my last response is HERE, but you should also do a search for RAM in the community articles for numerous other discussions and permutations of the subject.
To address your question more directly, “how much RAM” is right for you can only be determined by you and what you want to run. At face value you have more than enough RAM at present according to any number of articles on the subject, but the reality is that what you need will be determined by the size of the sample libraries and specific plugins you want to use. Two examples I regularly point to is the Berlin Strings library which has a recommended RAM size of 32gb to load the whole thing, and NI Pharlight/Ashlight/(and the other I can’t remember) which can run hundreds of voices and can bring my computer with 64gb of ultrafast ram to it’s knees.
In any case, your comment about loading times is almost certainly not related to RAM capacity, and if you are not having any audio dropout problems when everything is up and running you probable have no problems due to sample libraries and RAM capacity either. There are a number of things that you can do to improve your GP load time, that’s another subject that comes up often and has a lot of replies and suggestions and is covered in the User Manual (available at the “Docs” link in the upper right corner of this webpage) and Knowledge Base section of the GP website. Doing a search for GP load time in the Community articles will provide a wealth of knowledge that will almost certainly cover your experiences.