You are right. I have an AMD Phenom II X4 955, 4 GB of RAM and an AMD Radeon HD 6950 (the version with 2GB of RAM) and the game sometimes become very slow when you play in cities with more than one million inhabitants. Still, the performance difference is from heaven to earth if I compare it with my previous configuration (AMD 64 X2 5600+ and ATI Radeon HD 4850 512MB).
HoChiMin: I would recommend you to buy a more powerful graphics card (to notice a real difference, at least a HD 4850/HD 5750/6750 with 1 GB of RAM (don't buy a vcard with only 512MB) or a GTS 250/GTS 450/GTX 550Ti) and another module of 2GB of RAM (to complete 4GB of RAM).