At a guess, the problem is probably a timing issue with the way the antennas receive signals. At the speed you are using here, 2.4GHz or 5GHz, small things matter. Early tests with repurposed Part 15 hardware had problems initiating a link at 34 miles as the firmware designed for WiFi house networks (<100') had already given up waiting and was retrying before the ACK came back (IEEE 802.11 Experiments in Virginia’s Shenandoah Valley, QST July 2005).
John Champa, KBOCL (SK), did a lot of writing popularizing HSMM in CQ VHF magazine. The heyday of this seems to have been 2003-2008. While otherwise hard to get, scans of some of his articles are to be found at http://hsmm.wikispaces.com. In his Spring 2004 article called "The Frontllnee: HSMM Part 2 - Developments in Amateur Radio", he mentions:
Mixed Antenna Design Problem: This is the possibility of a unique
situation when using HSMM radio with certain antennas which requires
more amateur radio study. It is often thought that as long as both
antennas at the ends of an HSMM radio link are of broad-bandwidth
design (the HSMM signal is 22 MHz wide) and of the same polarization
(usually horizontal) then this is acceptable and all is okay. While
this may be true for other wide signals, such as analog ATV VSB
(vestigial sideband) signals or analog ATV-FM signals, it may not be
true with even more broad-bandwidth high-speed digital signals. Some
evidence seems to indicate that, for example, the use of a
horizontally polarized dish antenna with a dipole feedpoint at one end
ofthe link and the use of a horizontally polarized long loop Yagi at
the other end (again both broad-bandwidth antennas and both
horizontally polarized) may nonetheless introduce a problem (higher
BER, or bit error rate) because of symbol errors caused by the manner
in which each antenna manipulates the radio-signal wave front. Again,
further radio amateur experimentation with HSMM radio signals is
warranted to determine the impact of using mixed antenna types on the
radio link .
Unfortunately, he gives no specific references to his source on that an almost all the links mention in the acknowledge section are out of date.