It sounds like you may have estimated this encounter as a lot harder than it actually is due to the way CR calculations work.

An encounter calculator like Kobold Fight Club will tell you that this is a Hard encounter on the borderline of deadly, but it's actually barely past Medium. The reason is that when you apply the XP modifier based on the total number of monsters, you're supposed to ignore any monsters that are a lot weaker than the average - in this case, your thugs.

Encounter calculators aren't sophisticated enough to determine which monsters should be ignored since that's really a judgement call, so it'll apply a multiplier based on there being 6 monsters when it should be applying a multiplier based on just 2 monsters.

With encounter calculators, I usually just don't bother putting in any weak minion-type monsters, and just remember that the encounter will be slightly more difficult than whatever the calculator says.