I used to pray with the Prophet (may Allah's peace and blessings be upon him) and his prayer was moderate in length and his sermon was moderate in length

The Prophet's prayer and sermon were not overly long or brief. Yet, they were moderate, being performed and given by the role model for Muslims in general and for Imāms and preachers in particular. So, no one should cite this Hadīth as evidence that the prayer and sermon may take place very quickly, for the Prophet's guidance points to moderation, and sometimes prolongation that causes no boredom.