So how does this apply to computer science?
Well, a significant part of computer science research is determined—to a larger or smaller degree—by psychology in the informal sense. Computer systems are ultimately used by, or at least run for, humans; systems have to be designed with this in mind, and the goals of research ultimately come down to human factors. Outside of research entirely contained in one area—theoretical work meant to further other theoretical work; systems research answering questions to improve other systems—you expect most projects to be at least guided by some sort of human factors.
Of course, often the “guide” is pretty generic: we designed this system to be faster because humans subconsciously perceive a response time >100ms as slow. That’s a real psychological insight, but it’s pretty widely known—and the rest of the research, the stuff that actually takes people time on the project, will all be based on improving performance. In a sense, yes, psychology mattered; in another, it really didn’t.
Even when the work is more closely tied to human thought, it doesn’t have to relate to academic psychology or its research methods (just like art!) Programming language design is a wonderful example: it is fundamentally a design discipline, and while it can try to use empirical methods like psychology, it doesn’t have to—any more than visual design does. It can, just like visual design can use AB testing—but the best examples of design rarely involve extensive empirical testing, neither in programming languages nor in visual design.