If I remember correctly, you will receive Naturalization Document in the mail
from INS weeks after you sworn in as a US citizen. I am not sure if it is
possible to speed up that process. Search the INS web site yourself.
When you go to passport agency, you need the Naturalization Document
from INS to apply for the passport. By the way, your green card will be
taken by INS at the sworn-in ceremony.