You need to think of the wad files as neat little packages for the game to access and use. It is a bit like a *.zip or *.rar archive. You open it, and there are lots of files, but you only extract and use those you need.
The same goes for GP4. If your cars.wad has old texture files and such, then performance will not drop, because GP4 simply doesn't use them. Of course if you lack space on your HD, then you might want to do some spring cleaning from time to time in your wad files.
Btw, the size of your wad file will seldom bring a 32mb gfx card like GF2 ultra to its knees. You need to think about strong bus speeds on the motherboard, good ram and a strong CPU. Performance might drop if you insert much bigger textures and more detailed carshapes, but that is something else.
It's only after we've lost everything, that we are free to do anything.