You can improve the soil by adding organic matter (manure, bark) so even if you choose plants that are ok in clay they will still benefit by the soil being improved.
Hydrangeas are lovely but not usually great on the fragrance so you'd need to choose one known for its scent if that's important to you.
Other ideas - mahonia - lovely yellow fragrant flowers, evergreen so year round interest plus they flower in winter which is a massive bonus when everything else is dormant.
Philadelphus - white flowers, very pretty and fragrant, there are a few varieties, both single and double flowers.
Fuchsia - not fragrant but so pretty and colourful! Again many to choose from but they come as both shrubs and annuals, so be clear what you are buying.
Weigela - easy to grow shrub, don't think they're known for fragrance but nice flowers, again lots of different ones, some with gold/yellow/variegated leaves.
Nearly all shrubs will grow beyond a metre in time but most things can be pruned to keep small, but obviously you don't want anything that grows big fast, not buddleia for instance!
Have fun!