We considered it for DD1, whose birthday is the first week of November. I was one of the youngest in my grade (just made the cutoff by 4 days) and am very glad my parents didn't hold me back. I was bored a lot of the time in school anyway. It was lame to get my driver's license so much later than most of my classmates (particularly the redshirted boys!), but there were no real issues for me.
We ended up starting DD1 on time rather than early, for many of the reasons PP have mentioned. We homeschool, so it doesn't affect her nearly as much as a public/private school student, but we still didn't want her to be in high school youth group a year early, be the last to hit puberty, or be the youngest/smallest child in dance/sports/whatever trying to keep up with older kids. Thankfully homeschooling offers plenty of flexibility curriculum wise, so she is working on some 1st grade materials, though she is actually just in K. We're happy with our choice.
Allison
DD1 11/05
DS 04/08
DD2 11/11