The Dillinger file has revealed at least where the FBI was obtaining some of their weapons.  Simply said, they were borrowed but it appears payment was made for the actual ammunition.

In this instance, Dillinger and his gang were believed to be in the Indiana area in April, 1934 and E. J. Connelly, with Clyde Tolson's approval and assistance, borrowed multiple weapons from the Federal Laboratories in Pittsburgh, Pa.  These guns etc. were transported by 2 Pittsburgh Agents by train and delivered to Connelly. (The Bohemia incident was brewing at the exact same time.)

Two April, 1934 documents show the "borrowing process" and the details of the transport.

You can see one document here relative to a request by borrow weapons with regard to the anticipated problems the Agents were going to have with the Dillinger gang.  

You can see the delivery of those weapons to E. J. Connelly here.