So far as I know, nobody here is on the APRS Working Group, so it's difficult to say exactly what their motivations were, but I'll make a few guesses. First, Unicode was still fairly young in 2000, and personally I still have a horror of Windows code pages and other ugly mechanisms used to represent other character sets in hardware and software before Unicode came along. In those pre-Unicode days the printable ASCII characters, the ones represented by decimal numbers 32 through 126, were a sort of "lingua franca" understood by most hardware devices and operating systems, at least in the US. Many OSes and hardware devices couldn't tolerate character encodings that used more than one byte per character. Many OSes and devices used codes outside of the ASCII 32-126 range as control codes, so for instance if a file containing such codes was attempted to be printed, then one such code that happened to occupy a byte of the file might signal the printer to eject the page being printed and start a new page. Another such code might tell the printer to switch to its largest font. Such behaviors were device-, OS-, or software-specific, because for a long time there was no universal way to handle codes outside of the ASCII 32-126 range. Unicode existed in the year 2000, but its use was far from universal, and many OSes, devices, and software packages were completely ignorant of it.
I feel fairly certain that the motivation of the group was to minimize the confusion and incompatibility issues that were still common in the year 2000 with characters outside the ASCII 32-126 range. Also, all the members of the APRS Working Group had US call signs, and it's possible that some group members were somewhat ignorant of the need for other character sets by different languages and cultures.
Regarding how relevant the recommendation to use the ancient ASCII subset is today, I would say that how software and hardware should handle the user-defined data section is undefined. Many software packages still act strangely when asked to edit files with data of an unknown format in them, or print such files. So my advice would be to feel free to experiment, but be very careful to test as much hardware and software as you can before commonly sending packets with other character encodings in the user-defined data field out "into the wild".