In this talk, we will consider the hyperbolic Kac-Moody algebra associated to a certain rank 3 Cartan matrix and generalized Kac-Moody algebras that contain the hyperbolic Kac-Moody algebra. The denominator funtions of the generalized Kac-Moody algebras are closely related to certain modular forms. We will compute asymptotic formulas for the Fourier coefficients of the modular forms using the method of Hardy-Ramanujan-Rademacher, and obtain an asymptotic bound for root multiplicities of the hyperbolic Kac-Moody algebra. This is a joint work with Henry Kim.
