I just found a great answer to the problems all in one place in QST, September 2019. The author addresses the same problems in the question, and explains his fixes for a similar bridge-circuit-based Arduino analyzer originally published in QST, November 2017. Also this forum post lists the same problems and planned fixes, so it looks like the answer is that the orginal design had some faults, but they can be fixed.
Update - More information
After days of searching and hitting dozens of dead links, I have dug up some meaningful and linkable documents on the history and evolution of AD9850 Antenna Analyzers. It has been very time-consuming, but educational to find and organize all these as bookmarks and I want to share what I found.
NJQRP DDS Daughtercard Kit, site last updated in 2005, is the earliest mention I've found of a DDS measuring project, although in their page they link to AARL publication - Original DDSVFO Article by Curtis Pruess WB2V, QEX 1997, which was a PIC/AD9850 VFO, essentially the same circuit as an analyzer but without the RF bridge. The daughtercard version, however, included an RF amplifier for the DDS, addressing the low output level even at that time.
DDS-60 Kit from the AmQRP is a current web page with a 2014 upgraded version of the same daughtercard, with history and lots of detail about the daughtercard evolution.
Related to the above daughter cards is WØPCE "Sweep Gen" - Automated DDS Sweep Generator Measurement System that had a dead link to original article by Sam Green, WØPCE (in QEX for Nov-Dec 2008) but which by searching I found mention of a version II in QEX September/October 2012.
I couldn't find mention of early DDS projects used for an antenna analyzer until 2013. The first one seems to be Beric Dunn's K6BEZ Antenna Analyzer. After that, many other analyzers seem to be offshoots of his, including some using the Si5351 DDS. Beric's version did not use an RF amplifier, which I believe is one key problem prompting me to post my question.
Another related project is The Poor Ham's Scalar Network Analyzer (PHSNA) by Nick Kennedy—WA5BDU from 2014. It uses DDS, and also an AD8307 log detector (which I've read can be used instead of the diodes as an improved detector in the bridge.)
In conclusion, a website with files originating from 2016, Antenna Analyzer for Ham Bands by Jack Purdum (W8TEE), Farrukh Zia (K2ZIA) is their project that they published in QST, November 2017. Lots of documentation here! This one uses a nearly identical bridge circuit that Beric's did, but includes an RF amplifier for the DDS. This seems to be the most sophisticated and documented source of all, especially with the mods and customizing mentioned in QST, September 2019.
With the above being the best design articles I've found, I now have accumulated links of many other blogs and forums that have great discussions and ideas about all of this. If anyone posts a specific new question about this subject, I will watch and try to provide a good answer.