I screen usually my bacteria colonies by PCR, I just pick a part of the colony with a tip and put it in my PCR tube. No special buffer used, try that perhaps?