A report in the Mirror suggests that the Premier League duo have both tabled bids of £1.5m for the Ghana international, who was born in England but opted to play his international football for the country of his parents' birth.
The talented full-back, who can also operate as a winger, made 33 appearances in all competitions for the Tykes last season as the Yorkshire club finished in 14th place in the second tier.
The 25-year-old has now entered the final 12 months of his deal at Oakwell and would be able to leave for nothing next summer and, with that in mind, it is said that they will accept a cut-price sum for the player.
There is no indication as to which club Yiadom would prefer and it may well be that more enter the race as the summer progresses but, for now, the Swans are in with a shout.