Surely there is someone in the library system you can call to talk to, even if that particular branch is closed today?

Do your library cards go by numbers or by your name? If by some number, then ask them to cancel all cards in your name and issue you a new one. If by name, then maybe just cancel all cards in your name and put a tag on your account stating that some strong id (passport?) is required to issue a new one?

Worst case scenario - identity theft. I sincerely hope that is not the case, but might be worth checking bank records, credit card records and such too.