20.54Don't assume every non-diatonic chord is modulation
You hear E7 in C.
Then Am.
That's probably secondary dominant.
You hear Fm then C.
Borrowed iv.
You hear C♯°7 then Dm.
Passing diminished.
Use the chromatic categories we've learned before declaring a new key.