BakéGyamon (2006)

Blurb:
Sanshiro’s chances of having an adventure are slim to none in his tiny island hometown, until the day a mysterious stranger invites him to play a game... Without warning, Sanshiro is taken to a backwards universe to play BakéGyamon--a game pitting monsters against monsters. Along the way he meets other players who have a particular reason for being there--to obtain the wish that is granted to the winner. But how far can Sanshiro get when the monsters he’s been paired with are a bunch of little mud balls?!

Source: www.anisearch.com/manga/2764