I am a little over six feet tall, oddly proportioned, overweight, inflexible, with a couple shoulder injuries.
Everyone tries to stick me on a 58, but almost every 58 is too long in the top tube, forcing me to compensate with a micro-stem.
I generally fit on a 56 but need some odd combos of stem angle and spacer stacks.
But ... I just finished a 3.5 hour ride on my Dawes, which is a 50---Yes Fifty---cm frame.
Everyone everywhere would laugh at my clown bike with a foot of exposed seat-post, 35 mm of spacers, and a 120-mm 17-degree stem.
Thing is, it is the most comfortable bike I own. Not by much, and part of it is the $11 Vader saddle .... but even with rock-hard 23-mm tires it is the bike for the miles.
I don't care what the "experts" say. I listen to them, because some of them are pretty knowledgeable and experienced (that is, they have actually learned from their experience.) I take into consideration what experienced, intelligent people say ... but ultimately it is Me out there on the bike on the road 25 miles from home and it is Me who either does or does not feel pain.
On my various bikes I have swapped bars, saddles, stems, seat posts, spacers ... fine-tuned each of them so that I can enjoy riding them. No generic "expert opinion" counts for the miles of real-life testing I have done.
If your bike works for you and you can ride it without discomfort for a few hours .... sure, test different things if you like. But never forget that the baseline, the bike you have now, seems to be meeting all your needs. Don't let some "expert" talk you into doing something the "Right" way if it is wrong for you.
Also .... Measure your contact points on your current bike as it is currently set up. If that set-up works for you, you would likely want to export it to other frames.
Learn to read geometry charts. Plug in your current bike's dimensions. Compare and contrast.
Go to the Dollar Store and get a cheap "drafting kit"--- usually you can find a cheap plastic ruler, protractor, a 30-60-90 and 45-45-90 triangle and a compass for a buck, particularly at back-to-school time.
Get a sheet of graph paper and lay out your frame. Get the geometry chart online from the manufacturer, start at the BB shell, measure up for BB drop, draw a line parallel to the horizon, measure front/center and rear/center, make your marks, measure, and that should be the wheelbase. If not, refine your method.
Whether or not you have the wheels located you can do the seat tube, because you should have seat tube angle and length. You should also have virtual or effective top tube length, measured from the seat tube, which locates the top of the head tube. You have your head tube angle and length, so draw that in. Extends that down, and add trail at the base where it hits the wheelbase line, and that double-checks the front wheel position.
Also you can draw in the actual top tube and see if its scale length corresponds to the listed actual TT.
Take saddle set back and BB-to-saddle-center from your current bike and sketch in where the seat would be. Sketch in your stem (you know its dimensions) and then measure saddle-center to stem-clamp center on the drawing and see if it corresponds.
If you are a little careful it isn't hard to create a side elevation engineering drawing (a view of your bike from the side that shows the important aspects) which actually corresponds to the real bike. This is like a sixth-grade shop class assignment ... which means it was really hard for me, but I found it worthwhile.
Once I had the sketch, I could trace it onto another sheet, and change it to suit different frames. This way I could lay out a frame I was thinking of buying and see what I would need to make it fit the contact points I needed. I was able to calculate the seat post length, offset, stem length and angle, and spacer stack for my Workswell so I knew what parts to buy before the frame even arrived. It fit exactly as it was supposed to fit.
You can do this.
It matters because not every frame is the same ... as others have mentioned, head- and seat-tube angles vary, BB drop varies, and head tube length varies disproportionately with different sizes of the same frame..
I have found that a lot of frames don't gain a lot of top tube between 56 and 58 but gain a bunch of head tube, which increases stack and reach more than you'd imagine 2 cm of top tube would do.
I find all frames use different angles for head- and seat tubes throughout the size range, which means raising the seat a little on one size might mean moving it back more, which means changing saddle offset---if you didn't notice that, you might find your weigh too far back and your bars too far away compared to n the same frame in a larger size.
Also, if, like me, you have an other-than-standard body ... everyone will do their very best and still give you horrible advice. It just happens. They mean well ... bless their souls.
If your 54 frame with a 130 stem suits you ... end of topic.