Gee I sure do wonder. No idea who that could possibly be.

Joker

As much as I hate to admit it, Joker.

Personally, I believe Bane or Ra's al Ghul should be Batman's nemesis.

Joker or Deathstroke.

Who broke the Bat and who killed Alfred again?

The good old ... himself.

Nope. JK, obviously the Joker.

A list of things scarecrow has done: Given batman nightmares. Two-face: He's the holiday killer. But nobody knew that. Riddler: He does riddles and robs banks. Bane: Single-handedly broke batman killed his butler and enslaved his city. Joker: Killed robin, tricked the bat-family into thinking he cut off their faces, infected his entire city with a virus, took control of the justice league, killed batman, and has nearly driven batman to murder him, several times. Golly gosh gee, I wonder who it could be...

My hot take is that the cool thing about batman's rogues is that given the proper setup any one of them can be his nemesis. Joker gets focused on too much (obviously) and villains like Mr freeze, mad hatter, ivy etc all have the emotional connection and the ability to truly challenge batman. In my mind all of these guys are on the same level as joker, with no one truly being on top.

Joker was Batman's arch-nemesis.