The Judge adcom was probably using 640 because it's the lower point of the current cohort's middle 80% GMAT range:
http://www.jbs.cam.ac.uk/programmes/mba/apply/who-are-we-looking-for/The median GMAT is indeed 680, and while this is a good reference the median misses the point that business schools are looking for a range of people to fill their cohorts, not just those right in the middle of the spectrum.
Besides the GMAT, the adcoms will also be looking at (1) your undergraduate performance, (2) your work experience history and career progression; as well as (3) outside leadership experience, (4) your application essays + interviews, and (5) your references.
Perhaps you gave the adcom broad strokes on your work background and academic history and thought that you're relatively competitive on these other aspects, or just used the 640 as what s/he thought was the lower end of general competitiveness.