Food for thought.... This games uses little to no RAM to play. What it does use is a lot of cpu usage that fluctuates extremely. Even though you shut down animations if you have programs, processes, services, autoupdates running in the background then they take up valuable needed cpu loading & unloading space which can cause a slow down in loading as well as meter skips especially with dou or single core cpu's. So you need to either run programs like Game Booster3, Gboost, using the configure features, or run msconfig to shut off anything running that isn't necessary.
You could try increasing your virtual memory but I personally don't think it does much since this game isn't ram oriented. Then you mention using team speak which like skype will take up constant cpu usage during your games, which is ok but can slow the load time and cause more meter skips if hitting at high usage peaks or fluctuations.
I use a cpu meter so I know exactly how much of my cpu is being used and I can use that to guage when to use and release my swing meter.
There are other causes such as net type, whether you are using wifi or hardwired, wgt server switching/usage etc., all of that I believe can effect the load time. For the best game results you can obtain on your system it is imperative you have your system optimized to the best it can perform in addition to cleaning your temp, history and cache files periodically. Also you could try increasing your local storage in your flash settings to see if that helps as well as going to Macromedia.com and change your storage and security settings for your flash by deleting any unnecessary sites and increasing it for wgt.