$effect running into infinite loop

I'm trying to migrate from svelte4 to svelte5 and for that I would like to change some $: to $effect. A pattern that I used from time to time looks like that:

let {id} = $props();let thing = $state();let cache = new SvelteMap();$effect = (async () => {        if (!cache.has(id)) {                cache.set(id, await get_thing(id));        }        thing = chache.get(id);})

I like to update thing on id change and also whenever the cache entry is updated, because of that I like both to be reactive. However, I change cache (which is reactive) within an $effect, which I often read is a bad thing as it causes infinite loops.Is this a bad pattern? How else can I solve this?