Depends on where it is. Weight high up (like at the saddle) gets more involved in side-to-side rocking than weight lower down. Weight at the rims is also more noticeable than most places, especially at high speeds, due to its rotational inertia; on road bikes, the front wheel tends to have a bigger impact on feel than the rear, because it's on the steering axis. A five-pound weight duct-taped to the bottom bracket shell won't affect how a bike feels very much at all.
Overall, a five-pound gain generally doesn't doom a road bike to losing its liveliness. My 1983 Miyata 710 has a similar tight and punchy "feel" as my Emonda ALR 5, despite weighing about 5 pounds more... the two bikes have very similar steering geometry.

As far as performance effects? Mostly your speed will take a couple-percent hit or so on steep climbs, depending on bike and rider weight.

If you’re anything like me, you’ll probably notice the 5lbs you’re carrying around the guts more!

But getting the best bike you can is always worthwhile - pushes us to live up to the bike!

I now have a 15 lb road bike, I don't really notice the difference in weight from my older 19 lb bike.

I have noticed a difference in the wheels I chose. I went from a Alex DA28 rim with 105 hubs to a set of HED Ardennes.

The old rule of thumb was 4s/lb in a flat 40k if you could ride 40k/hr. So 5lb would be 20s in a 40k or 90s in 112 miles. But this is based on it being flat! At less than 1s/mile, there's no doubt in my mind that most of us couldn't feel the difference.

For pure ascent speed, not more than focusing on body weight.

Now, handling.....yeah you'll feel it.

An old steel road bike with a very slack setup will be like turning a ship compared to a light modern buzzy carbon racer.

Especially when laying it over in "S" curves where you toss it left and right.

Modern road race geometry was already fairly mature by the end of the vintage steel era. And if you go way back to when the angles tended to be slacker, many bikes used tons of fork offset, and the steering was actually pretty light, even if a bit odd.

If you want to experience a bicycle that handles like a boat... convert an early 1980s mountain bike to narrow road drop handlebars, and pump the tires low enough that the pneumatic trail gets real aggressive.

HED Ardennes came on my new bike last year. They are awesome. Some of the smoothest wheels I've ever had.

If you are upgrading from a 2010 aluminum bike to a new bike in the 15 lb. range then you will likely notice more than just the weight savings. A carbon frame is almost certainly going to be more smooth and noticeably absent of the high frequency buzz that is felt on a metal bike, while being quite a bit stiffer which is definitely felt when putting power to the pedals. Assuming you are riding a 2010 wheelset, you will also notice a big difference in comfort and handling by moving to a wider internal rim width (21mm-ish) and tire (25c-28c).

I could be wrong, but I think bikes weighing as little as 15 lbs. still command a hefty premium as they typically represent the lightest carbon layup. You can save a ton of money and often end up with a better bike by purchasing the slightly heavier carbon frame (maybe 100g-200g heavier) and putting the money you save in what really matters, i.e. upgrading wheels, tires, crankset, and cassette. There is a ton of low hanging fruit in these components when it comes to saving weight off of a factory build, and the upgrade(s) will be far more noticeable than a lighter frame.

I always remember what someone said when they crunched the numbers for Alpe D'Huez, which is a 1100m/3500ish foot climb. One pound will slow you down by 14 seconds. So if you are climbing the Alpe then five pounds is one minute 10 seconds slower.

Buy aero wheels before light weight. People say that light wheels 'feel' snappier, but the math does not support them being faster. Whatever extra energy you use to spin up a heavier wheel is not lost. It rolls longer faster.

I was really curious if I could even tell a 10 pound difference on a rolling course. I could not. Same wattage, same time. That is not to say that it might not be minute fractions faster, but you would have a hard time testing it on your own.

Sometimes the math is missing other variables from real life. I've been doing one my local fast rides and did some swapping between zipp 404s and zipp 202s (over 6 runs of the same route), everything else being the same (bike, tires, etc). With the 404s I was 20 watts average higher on the climbs, HR was near my max and get dropped by the 3rd climb. With the 202s I had no problem staying on for all the climbs and HR was about 7-10 bpm lower at the highest point.

Despite what the math says, I even find the 202s really close to the 404s when on the front doing 30+mph. Maybe if everything was pancake flat and the wind always came at an ideal angle it would be different but I ride near the shore and get shifting crosswinds throughout.

Please explain why the Zipp 404s make you produce 20 more watts. It just doesn't make any sense. So if I swap out my aluminum training wheels for the Zipp 404s, do my power output instantly increases by 20 watts on the hills without any additional training, like taking EPO? In other words, do the Zipp 404s magically raise my FTP by 20 watts?

That is the problem with doing something and then making a conclusion without isolating every other variable. You say everything was the same equipment-wise and then admit that you get shifting crosswinds throughout. How can you possibly trust any data that has consistently variable wind? I do a lot of outdoor testing and some of it is very frustrating because I know it is junk. When I pull out a piece of equipment and I suddenly ride phenomenally fast, my first thought is to go grab my old stuff and ride it quickly to see what was up. If something seems odd in testing (like 202 being 20 watts faster than 404s) then it probably is.