We fixed the bug introduced by the patch for managing the shared
clocks at the commit 809f44a0cc ("ALSA: usb-audio: Clear fixed clock
rate at closing EP"), but it was merely a workaround. By this change,
the clock reference rate is cleared at each EP close, hence the still
remaining EP may need a re-setup of rate unnecessarily.
This patch introduces the proper refcounting for the clock reference
object so that the clock setup is done only when needed.
Fixes: 809f44a0cc ("ALSA: usb-audio: Clear fixed clock rate at closing EP")
Fixes: c11117b634 ("ALSA: usb-audio: Refcount multiple accesses on the single clock")
Link: https://lore.kernel.org/r/20220920181126.4912-1-tiwai@suse.de
Signed-off-by: Takashi Iwai <tiwai@suse.de>