We present an exact Bayesian inference method for inferring posterior distributions encoded by probabilistic programs featuring possibly \emph{unbounded loops}. Our method is built on a denotational semantics represented by \emph{probability generating functions}, which resolves semantic intricacies induced by intertwining discrete probabilistic loops with \emph{conditioning} (for encoding posterior observations). We implement our method in a tool called “Anonym”; it augments existing computer algebra systems with the theory of generating functions for the (semi-)automatic inference and quantitative verification of conditioned probabilistic programs. Experimental results show that “Anonym” can handle various infinite-state loopy programs and exhibits comparable performance to state-of-the-art exact inference tools over loop-free benchmarks.