Physical random numbers generated by using quantum measurement are in principle impossible to predict.We have demonstrated generation of physical random numbers by measuring quadrature amplitudes of vacuum states with a broadband homodyne detector.In this method, random numbers can be generated at 500Mbps.By multiplying a toeplitz matrix to the generated numbers, we generated physical random numbers that passed all the test of TestU01 crush test.