Is it stupid to date a guy that's 20, when i'm 26?

I'm totally infatuated with this guy I work with.. he's tall, handsome, sooo good looking... i just want to jump him. He's been hinting about us dating though, and I never really considered it before. It's so foreign to me because i've always dated older men. I seriously get a huge smile on my face when he walks by. This is the same feeling I had the last time I fell in love, but then I think it's just because he's so young and hot. He gives me erflies. Am I just infatuated, or could this be something serious?