No. “I only waited five minutes” is normal English. Readers will usually understand that the wait lasted just five minutes. “Only” often appears before the main verb.
“I waited only five minutes” puts the limit directly beside the duration. That version can help when the surrounding sentence leaves room for another interpretation. See our only placement guide for more examples.
Related question: Does putting “only” before the subject change the meaning?
Reference: Cambridge English Grammar Today: Only.
