This quotation requires some historical background for its context.
Alexander Hamilton (who would become Secretary of the Treasury under President George Washington, and whose portrait adorns the $10 bill) was speaking  in New York’s ratification convention, the assembly elected to debate whether New York should ratify the newly proposed federal constitution that is, our current federal constitution.
The Articles of Confederation, which was then the compact that united the 13 states in one government, conferred on the Congress no power to tax. The Congress could only requisition funds from the states. Some states were forthcoming, but others were not and were very much in arrears.
But the Confederation Congress could borrow money. And that’s what it did to meet expenses.
Hamilton was arguing that Congressional authority to requisition only, and not to tax, was putting the young country in a very precarious situation: the Confederation Congress had to borrow more money to keep up with payments on the money it had already borrowed. And, he was arguing that giving a newly-constituted Congress the power to tax would rescue the country from that situation.
But there we are, again, just as we were two and a quarter centuries ago.