In case module is removed dynamically with ongoing playback, during
module cleanup codec state is mistakenly modified. State should be
modified for module only. Fix this.
Fixes: 76414cb499b7 ("audio: Use single codec driver
registration")
Signed-off-by: Vaibhav Agarwal <vaibhav.agarwal@linaro.org>
Reviewed-by: Mark Greer <mgreer@animalcreek.com>
Signed-off-by: Alex Elder <elder@linaro.org>