In the early 2000s, with Extreme Programming's high focus on testing as a critical aspect of software development, many of us were introduced to or became used to applying specific testing patterns that today are considered anti-patterns. Back then, some were not seen as bad, but often the reason was that we really had no other choice, as you mainly dealt with closed-source libraries and frameworks. Other than inside Stack Overflow, I find it hard nowadays to find some articles mentioning the topic, so here goes my contribution.
Testing Private Methods
You shouldn't do it [1]. Your public methods represent your class surface/API/interface, and private methods are implementation details; so, when testing private methods, you're coupling the test to the internal implementation details, which should be free to change with as less friction as possible.
Instead, do one of the following:
- Refactor some of the private logic to a class, and use object composition: So you test the logic in isolation and can use a mock when testing the class that now will instantiate the refactored code
- Focus on testing indirectly: Your goal is not 100% code coverage; your goal is testing an action, a behavior, or a concept. Focus on that and not on checking every tiny detail. Or else apply the previous point
With some languages having either poor or no encapsulation, it becomes very appealing and an easy way to "speed writing tests", but you should remember that you're breaking object-oriented encapsulation: If the method was private, it was meant not to be used directly from the outside, not even from a unit test.
In the past, we relied on either Reflection to access some private methods, or inheritance and polymorphism (when the language had good enough support) and created a child that exposed public methods to ease testing and/or mocking. But today, I advise against this and instead go for wrapping the external class and testing its public surface only. Most, if not all, scenarios can be covered by composition.
And, as a side note, for scenarios like Javascript module exports, where access modifiers are only either exported or not exported at all (and anything not exported can't be tested via the module itself), there are specific techniques, like the Testables Named Export pattern.
Mocking Class Under Test (CUT) methods
Sometimes mentioned as "System Under Test", both represent the same wrong concept: You should never mock your any class methods of the main class you're testing in a test. If you need to do it, or think that doing so would simplify the tests, that's a clear signal of a refactor waiting to materialize: refactoring to another method or a different class.
There's really not much to it: If your class does A and B, and B is a private method only called from A, either you test everything when testing A (maybe ignoring the fact that you know there's a B method), or you extract B to a separate module/class, where it is ok to test it in isolation, and make A instantiate B, and then mock B when testing A.
Final Thoughts
I've heard at times some pushback comments like "but I shouldn't rewrite my code to conform to tests". While that point can theoretically be correct, what in practice happens is that testing often surfaces problems in your existing code. It is not the cause of why you need to change your code, it is helping you identify the changes you that need to be done.
If your code were simple, then it would be easy to test.
In my opinion, tests should aim to reproduce production conditions. We are already mocking, stubbing, and faking so many things (at times maybe too many); thus, we shouldn't do yet more shortcuts.
[1]: Sample reference: Unit Testing Principles, Practices, and Patterns book