First, his plank of Hindutva has unified various sections of the community and second, the economic prosperity that ushered in under his dispensation has benefitted every community. Under Modi's rule, Gujarat has seen emergence of a very large middle-class that is comfortable with Modi's economic policies. And this new middle class consist of each and every community.
More than Congress, it's the internal machinations than can spoil Modi's prospects. The biggest challenge to Modi will come from Keshubhai Patel who used to be his mentor once upon a time.
After being marginalized in the party affairs for a long period, Keshubhai finally broke ranks with BJP and has floated his own party – Gujarat Parivartan Party. Keshubhai is a formidable Patidar(Patel) leader and he has made it clear that this time he will leave no stone unturned in ensuring Modi's ouster.
Even in 2007, Keshubhai wasnot happy but Modi is believed to have exploited differences between Leuva and Kadva Patels. Keshubhai belongs to Leuva Patel community but this time he is trying to unite all Patidars under his new found party.