As we know, Swashbuckler isn't that attractive compared to other classes. It has abilities tied to dexterity (Grace, Elegant Strike, Swashbuckler Dodge, Feint, Second Intention) or which require wearing light armour, and is clearly designed with Dex as main stat in mind (Uncanny Dodge, Improved Evasion, Epic Dodge as a feat option). However, it cannot make proper use of Dex due to not being included in the Rogue feats Stealth Bonuses, Lightly Armored and Specialty Weapons. You can dip into Rogue, but the Swashbuckler abilities need deep levels, which means you don't get synergy and don't get the real benefits of those Rogue feats like you do with a Rogue/SD build for example.
I suggest simply adding Swashbuckler to the list of classes which count towards Stealth Bonuses, Lightly Armored and Specialty Weapons synergy.