E. Cobham Brewer 18101897. Dictionary of Phrase and Fable. 1898.
Hand over; pay down; stand treat. Fingers are called forks, and this may suffice to explain the phrase; if not, we have the Anglo-Saxon verb feccan (to draw out, to take), and fork out would be fec out.