Do my friendships grow, even if I cannot progress the story?

At certain points in my playthrough of Three Houses, I've earned enough support to have the next support level/conversation happen with a character, but was unable to, with the message "It will take some time before you can deepen this bond". This appears to be because some support conversations can't happen until the story progresses, else they wouldn't make sense.

However, during this time that you cannot increase the support level, you can seemingly continue earning support with the character. Any activity that would provide support (gifts, tea, lost items, assistance in battles, etc.) continue to show the heart icons indicating your support grew.

Does all that affinity earned while unable to view the next support level apply to the subsequent support level, or is it lost? Asked another way... if I earned enough support with someone to view their B support conversation, but am unable to view it until later in the storyline... and I continue to earn support in the meantime by showering that character with gifts and lost items, does all that support contribute to making me closer to the A support conversation when the story allows me to finally view their B support?