<p>Is it a good idea or do I need to calibrate my E steps after I switch to a new filament due to the different types I use (to make my prints accurate)?</p>

<p>E.g. calibrate when switching from PLA to ABS/PETG? </p>

No that will not be necessary.
However, you could use calipers to measure the diameter of the filaments (e.g. at 5 positions over a few meters) and calculate the mean diameter, if there is a significant difference between the new and the currently used filament you could change the diameter in the slicer (or the flow modifier), you do not need to calibrate the steps per millimeter every time you change filament.
You only need to calibrate the steps per millimeter if you change something in the extruder hardware setup, e.g. different extruder, different stepper driver, a new gear, etc. As long as the hardware is not changed a calibrated extruder setup will move a certain amount of filament regardless of the diameter variation (per rotation of the extruder gear an amount of
$2 \times \pi \times (gear\ radius)$
mm of filament.