I agree with the above. I'll use Final Fantasy VII and Xenogears as an example. Both games have the possibility of 9 playable characters.
Final Fantays VII:I personally felt the party management was done poorly in that game. You had characters join up through story sequences that felt out of place and tacked on when they should have been side characters, and one of the side characters (Vincent) felt like he should have actually been a required party member. Not to mention, I didn't care for half the characters in that game (Yuffie and Cait Sith were the worst, in my opinion).
Xenogears:Then you have Xenogears, and while a couple of the characters were near useless in battle, most of them had a reason for being there, and were built around the story in such a way that made sense.
So it doesn't matter how many you have so much as how you choose to develop them and make them part of the story. If you're just adding playable characters just for kicks and to create a long mass of characters, more than likely you're not going to find people interested long enough to care to play through the game. If you're creating characters out of careful planning and consideration, it doesn't matter how many you have (3 or 10), as long as people are invested into the story and characters enough to care about what happens to them.