The Allez is in the 'competitive' category, but that doesn't necessarily mean you have to race it. They also have an endurance category that includes the Secteur, which has geo modeled after the Roubaix.
The categories mentioned are really just Specialized terms for race and relaxed geo, that you may have heard about. Generally speaking, race geo allows for a slightly more aggressive rider position with quicker (some say twitchy) handling. Relaxed geo bikes allow for a slightly more upright position, and slightly slower (more predictable) handling.
Also generally speaking, all else being equal, bikes with longer wheelbases (relaxed geo) will have a somewhat more comfortable ride. Given your mentioning NYC streets, if comfort is high on your priorities, the Secteur may be more to your liking, but IME setting the correct tire pressures for rider and conditions goes a long way in contributing to a cyclists comfort.
Bottom line is that you have to visit your Specialized dealer, get sized/ fitted and head out on some test rides. Get out on the roads and focus on fit/ feel, ride and handling. IME that's the best way to determine your preferences and whittle the field.
Lastly, while I'm admittedly a Spec fan, this being your first road bike, I suggest you visit some other reputable shops and try out some difference brands/ models. The exposure to subtle differences may lead you back to Specialized, or it may make you realize there's something that just feels right about another. You'll likely have this bike awhile, so it's worth the time and effort to get it right.
