Without seeing the exact plan for you deck it not possible to say with certainty. The wider (deeper) the joist the more load it can support with deflecting. In other words, when you put weight on the deck, the joist won't bend. The usual solution for dealing with undersized joists is to use more of them, spaced closer together. So, for example, instead of 16" spacing with 6" joist, you may have to go to 12" spacing with a 4" joist. You should really have you plans looked at by a qualified professional.
Understand that I'm not an engineer (nor do I play one on TV) but I would suggest adding another beam so that span of the 2x4 joists is under 6'. The general rule of thumb is maximum span is 1½ times the depth of the joist in feet (so 2x6 can span about 9' but that does vary depending on the type of lumber used). I'd also up the foots to 4 per beam to drop the span there to 4'.
Some good information on deck construction from the American Wood Council can be found here.