Is the phrase "savoring the taste of contentment" correct?

That is correct, but it stretches it personifies the word "contentment," which may be what you are going for. If that's the case, bravo. You could also just say "savoring the contentment" without "taste of."