Other Rightly-Guided Caliphs?

In reference to the hadith that commands us to follow the sunnah of the prophet (saw) and the sunnah of the rightly-guided caliphs, how did the Muslim scholars come to the conclusion that this only refers to Abu Bakr, Umar, Uthman, and Ali (with the possible addition of Umar ibn Abdul Aziz). Since the hadith didn't name specific caliphs, isn't there a possibility that the sunnah of pious caliphs in later times should be followed as well? In the history of the caliphate, aren't there at least a few caliphs after the first four that can be said to have been rightly guided?

The above questions provide a good illustration why the Qur'an and the Sunna are not read in a vacuum or without context. Rather, we understand them, especially the foundational verses and reports with regard to jurisprudential first principles, by paying the utmost attention to all the other evidence that brings to bear on the issue including other verses, other hadiths, and then ONLY in the understanding of the Umma, i.e. the qualified Ulema, especially when history confirms all of the above as it does in this specific issue. Very briefly, that the Prophet, upon him blessings and peace, DID name the first four specifically, especially the first two; that he also named very few others besides them, such as Ibn Mas`ud; that he specifically recommended "his own qarn (century or generation), then the one that follows it, then the one after that" and then disrecommended whatever follows; that he confined the rightly-guided caliphate to the Prophetic model that was to follow him directly without lasting much, until its return at the end of time etc.

And Allah knows best.

Was-Salam.

Hajj Gibril