Short answer
No. We know of no authentic hadith that says so. The sound reports say that leadership will remain with Quraysh. That is the whole tribe, not only the Prophet's (ﷺ) family, and Abu Bakr, Umar and Uthman were all of Quraysh. The sayings quoted for the family are real and dear to us. "Of whomever I am the mawla, Ali is his mawla" calls for love and support. "The Book of Allah and my household" asks us to care for his family. Neither names a ruler. The community chose Abu Bakr, and Ali pledged to him. The family's honour is real. The question here is only about leadership.
Full explanation
What the sound reports say about leadership
The Prophet (ﷺ) said that the caliphate will remain among Quraysh, even if only two people are left. He also said that people follow Quraysh in this matter. Quraysh is a tribe. The Prophet's family is one clan within it. When the Ansar, the Muslims of Medina, met to choose a ruler, Abu Bakr spoke of Quraysh as the best family among the Arabs. Mu'awiya also reports the Prophet (ﷺ) saying that this authority will remain with Quraysh, as long as they uphold the religion. The reports name a tribe, not a house.
A report from Abdullah ibn Umar that the Messenger of Allah (ﷺ) said: the caliphate will remain among Quraysh, even if only two people are left.
Abu Hurayra reports that the Messenger of Allah (ﷺ) said: people follow Quraysh in this matter; the Muslims among them follow the Muslims among Quraysh.
Aisha reports that when the Prophet (ﷺ) died, Umar insisted he was not dead. Abu Bakr came, kissed the Prophet's face, then told the people that whoever worshipped Muhammad should know Muhammad was dead, and whoever worshipped Allah should know Allah is alive and never dies. He recited 39:30 and 3:144, that Muhammad is no more than a messenger, and the people wept.
The same entry as 3667, which gives the scene at the shed of Banu Sa'ida. Some of the Ansar, the Muslims of Medina, said there should be one leader from them and one from the emigrants. Abu Bakr answered that the emigrants of Quraysh were the best family among the Arabs, and told them to choose Umar or Abu Ubayda. Umar answered that they chose Abu Bakr, their chief, the best of them and the dearest of them to the Messenger of Allah (ﷺ), took his hand and pledged, and the people pledged after him.
Muhammad ibn Jubayr ibn Mut'im reports that Mu'awiya, told that Abdullah ibn Amr was saying a king would come from Qahtan, said he had heard the Prophet (ﷺ) say that this matter, the rule, is in Quraysh, and that God throws down on his face whoever opposes them, as long as they uphold the religion.
Muhammad ibn Jubayr reports that Mu'awiya said he had heard the Messenger of Allah (ﷺ) say that this matter, the caliphate, will remain with Quraysh, and that Allah will throw down on his face whoever rebels against them, as long as they keep to the rules of the religion. This is the report in the chapter on judgments.
"Of whomever I am the mawla, Ali is his mawla"
This saying is real. It is in Jami al-Tirmidhi and Sunan Ibn Majah. The word mawla means a close friend, a helper, a protector. In the Qur'an the same word is used of Allah (﷾) as the Prophet's (ﷺ) mawla, and Yusuf Ali renders it as Protector. Another sound report shows the Prophet (ﷺ) defending Ali when Buraida disliked him: he asked whether Buraida hated Ali, and said Ali deserved more. Other reports place the saying at Khumm, on the way back from the Farewell Pilgrimage. Wherever it was said, it calls for love of Ali and support for him. It does not say that Ali is the ruler after the Prophet (ﷺ).
The Prophet (ﷺ) said: of whomever I am the mawla, Ali is his mawla. Mawla means close friend, helper or patron.
Sa'd ibn Abi Waqqas defends Ali when Mu'awiya criticises him. He recalls three sayings of the Prophet (ﷺ) about Ali: of whomever I am the close friend (mawla), Ali is also his close friend; you are to me as Harun was to Musa, except that there will be no prophet after me; and I will give the banner today to a man who loves Allah and His Messenger.
“If ye two turn in repentance to Him, your hearts are indeed so inclined; But if ye back up each other against him, truly Allah is his Protector, and Gabriel, and (every) righteous one among those who believe,- and furthermore, the angels - will back (him) up.”
Buraida reports that the Prophet (ﷺ) sent Ali to bring the fifth of the booty, and that Buraida disliked what he saw and told the Prophet (ﷺ). The Prophet (ﷺ) asked whether he hated Ali, and said Ali deserved more than that from the fifth.
"The Book of Allah and my household"
Zayd ibn Arqam reports that the Prophet (ﷺ) spoke at a watering place called Khumm. He said he was leaving two weighty things. The first is the Book of Allah (﷾), with right guidance and light, and he told them to hold fast to it. The second is the people of his household, and he said: I remind you of your duties to the members of my family. Zayd explained that they are those to whom charity is forbidden. The words ask for care, love and honour. They do not speak of rule.
Zayd ibn Arqam reports that at a watering place called Khumm, between Mecca and Medina, the Prophet (ﷺ) praised Allah and said he was leaving two weighty things among the people: the Book of Allah, which holds right guidance and light, so hold fast to it; and the people of his household, reminding the people of their duties to the members of his family. Zayd said the Prophet's (ﷺ) wives are of his household, but the household meant here are those to whom charity is forbidden: the family of Ali, of Aqil, of Ja'far and of al-Abbas.
"As Harun was to Musa"
The Prophet (ﷺ) left Ali behind in Medina when he marched to Tabuk. Ali asked if he was being left behind with the women and children. The Prophet (ﷺ) answered that he should be pleased to be to him what Aaron was to Moses. He added that there would be no prophet after him. It is a great honour, said for that occasion. Moses also left Aaron in charge of his people while he went up the mountain (7:142), as the Prophet (ﷺ) left Ali in Medina during Tabuk. It does not speak of rule after the Prophet (ﷺ).
Sa'd ibn Abi Waqqas reports that the Messenger of Allah (ﷺ) left Ali behind in Medina when he marched to Tabuk. Ali asked: are you leaving me behind with the women and children? The Prophet (ﷺ) answered: are you not satisfied to be to me what Aaron was to Moses, except that there will be no prophet after me?
“We appointed for Moses thirty nights, and completed (the period) with ten (more): thus was completed the term (of communion) with his Lord, forty nights. And Moses had charged his brother Aaron (before he went up): "Act for me amongst my people: Do right, and follow not the way of those who do mischief."”
The community chose, and Ali pledged
On the day the Prophet (ﷺ) died, the Muslims met to choose a ruler. Umar took Abu Bakr's hand and said: we choose you, our chief and the best of us. He added that Abu Bakr was the dearest of them to the Messenger of Allah (ﷺ). The people pledged after him. About six months later Ali sent for Abu Bakr, then praised his right in public and gave his pledge. Ali also named Abu Bakr and then Umar as the best after the Prophet (ﷺ). Whatever hurt was felt, Ali himself closed the matter.
Aisha reports that when the Prophet (ﷺ) died, Umar insisted he was not dead. Abu Bakr came, kissed the Prophet's face, then told the people that whoever worshipped Muhammad should know Muhammad was dead, and whoever worshipped Allah should know Allah is alive and never dies. He recited 39:30 and 3:144, that Muhammad is no more than a messenger, and the people wept.
The same entry as 3667, which gives the scene at the shed of Banu Sa'ida. Some of the Ansar, the Muslims of Medina, said there should be one leader from them and one from the emigrants. Abu Bakr answered that the emigrants of Quraysh were the best family among the Arabs, and told them to choose Umar or Abu Ubayda. Umar answered that they chose Abu Bakr, their chief, the best of them and the dearest of them to the Messenger of Allah (ﷺ), took his hand and pledged, and the people pledged after him.
In a long report from Aisha, Ali, who had not given his pledge during the six months of Fatima's life after the Prophet's (ﷺ) death, seeks reconciliation with Abu Bakr once she has died. He sends for Abu Bakr, and Abu Bakr comes to him. Ali says the family know Abu Bakr's excellence and what Allah has given him, and do not envy it, but that Abu Bakr had not consulted them, and they had thought they had a right in the matter because of their kinship to the Messenger (ﷺ). Abu Bakr weeps and says that kindness to the relatives of the Messenger (ﷺ) is dearer to him than kindness to his own. Ali gives him his pledge of allegiance that afternoon and praises Abu Bakr's right in public. The Muslims are glad and draw close to Ali again. The same report begins with Fatima's request about her inheritance; that part is discussed on the page "Was Fatima angry with Abu Bakr?".
Ali's son Ibn al-Hanafiyya asked his father who the best of people were after the Messenger of Allah (ﷺ). Ali answered Abu Bakr, then Umar. Afraid that Ali would name Uthman next, he asked: then you? Ali replied that he was only a man among the Muslims.
What Ali and al-Hasan did
When the people came to Ali after Uthman was killed, al-Tabari records that he at first declined, and accepted only when the people pressed him. When he ruled, he asked the people of Iraq to carry on as before, because he disliked disagreement. His son al-Hasan later handed leadership over to unite the Muslims. The Prophet (ﷺ) praised in advance the peace by which his own grandson gave up rule for the sake of unity.
Al-Tabari records that after Uthman was killed the people thronged to Ali and asked to pledge to him, saying they feared for Islam. Ali at first declined, and the people raised their voices and urged him until he accepted.
Describes how, after Uthman was killed, a delegation of the Muhajireen and the Ansar asked Ali to accept the caliphate and he refused. The companions sent the delegation back to press him harder, and he accepted on conditions of his own. The Muhajireen, the Ansar and the people then took the oath of allegiance to him. Razwy says the community of Muhammad (ﷺ) "drafted" Ali, and quotes Gibbon that he declared he had rather serve than reign.
Ubayda reports that Ali told the people of Iraq to judge as they used to judge, because he disliked disagreement, until the people united as one group or he died as his companions had died.
Al-Hasan al-Basri reports that al-Hasan ibn Ali led armies like mountains against Mu'awiya, and Amr ibn al-As warned that they would not turn back until they had killed their opponents. Mu'awiya asked who would be left to care for the people, their women and their children if the two sides destroyed each other, and sent two men of Quraysh to offer peace. To every condition al-Hasan asked for they answered that they would stand guarantee for it, and al-Hasan made peace with Mu'awiya. The narrator then recalls the Prophet's (ﷺ) words from the pulpit about al-Hasan making peace between two great groups of Muslims.
Honour is not rule
The family's honour does not depend on rule. The Prophet (ﷺ) loved al-Hasan and asked Allah (﷾) to love him. Abu Bakr told the people to honour Muhammad (ﷺ) through his family. The Prophet (ﷺ) named Ali as a man who loves Allah (﷾) and His Messenger. Muslims owe the family love, care and respect. They also believe, as the Prophet (ﷺ) foretold, that a man of his family, the Mahdi, will be raised up near the end of time.
Al-Bara reports that he saw the Prophet (ﷺ) carrying al-Hasan on his shoulder and saying: O Allah, I love him, so love him.
Abu Bakr said: watch over (honour) Muhammad (ﷺ) through his family.
Salama reports that Ali had stayed behind at the time of Khaybar because of eye trouble, then set out after the Prophet (ﷺ). On the eve of the conquest of Khaybar the Prophet said that the next day he would give the flag to a man who loves Allah and His Messenger, and that Allah would grant victory under his leadership. Ali arrived unexpectedly, the Prophet gave him the flag, and Allah granted victory under his leadership.
Ali ibn Abi Talib reports that the Prophet (ﷺ) said: if only one day of this world remained, Allah would raise up a man from my family who would fill the earth with justice as it has been filled with oppression.
Perspectives
Some Muslims believe the Prophet (ﷺ) appointed his family to lead
Some Muslims, from love of the Prophet's (ﷺ) family, hold that he named Ali, and then his descendants, as the only rightful leaders. They bring these arguments.
"Of whomever I am the mawla, Ali is his mawla" appointed Ali as leader.
Rebuttal
Mawla means close friend or helper, and in 66:4 the same word describes Allah's (﷾) protection of the Prophet (ﷺ). The sound reports call for love of Ali. They do not mention rule. Ali himself later pledged to Abu Bakr and named him the best after the Prophet (ﷺ).
Some reports add that just before the mawla saying the Prophet (ﷺ) asked: am I not closer to the believers than themselves? Qur'an 33:6 says he is, so the saying gives Ali the same authority.
Rebuttal
The closeness in 33:6 is the Prophet's (ﷺ) own rank as the Messenger. What he asks for Ali is love and support: when Buraida disliked Ali, he said Ali deserved more. And Ali himself later pledged to Abu Bakr and named him the best after the Prophet (ﷺ).
"The Book of Allah and my household" makes the household the leaders, beside the Book.
Rebuttal
In the report the Prophet (ﷺ) said: I remind you of your duties to the members of my family. That is a call to care for them. He told the people to hold fast to the Book of Allah (﷾).
"Hold fast to both and you will not go astray" makes following the household as binding as following the Book.
Rebuttal
Zayd counts the households of Ali, Aqil, Ja'far and al-Abbas as the family. Holding to them means following their way, and Ali's way was the pledge and unity. In the same report the Prophet (ﷺ) said: look at how you treat them after me. That is a call to care for them.
"As Harun was to Musa" made Ali the Prophet's (ﷺ) successor, as Aaron was Moses's.
Rebuttal
It was said when the Prophet (ﷺ) left Ali behind in Medina during Tabuk, as Moses left Aaron in charge while he was away (7:142). It ends with the words: except that there is no prophet after me. It does not mention rule afterwards.
Leadership must remain in the Prophet's (ﷺ) family, and the Mahdi from his family shows it.
Rebuttal
The Mahdi reports speak of the end of time. Al-Hasan handed leadership over to unite the Muslims, and the Prophet (ﷺ) praised that peace in advance.
The Islamic perspective
No authentic hadith says that only the Prophet's (ﷺ) family could lead. The sound reports name Quraysh. The family's honour is real, and Ali himself pledged to those before him and took office by the community's pledge.
The sound reports say authority and the caliphate remain with Quraysh, the whole tribe.
The mawla saying and the saying about the Book and the household call for love and care, not rule.
The community chose Abu Bakr, and Ali gave him his pledge.
Al-Tabari records that Ali at first declined the pledge, and accepted when the people pressed him.
Al-Hasan handed leadership over to unite the Muslims, as the Prophet (ﷺ) foretold.
Muslims love and honour the Prophet's (ﷺ) family, and believe in the Mahdi of the sound hadith.