There is a difference between Self Defense and fighting. In a fight. the objective is to overcome your opponent. In self defense, your objective is to survive. It is important to know that if someone attacks you they will probably be stronger, larger, faster, and it is important to assume some ability on their part.
Often times they may be competent fighters or you may be dealing with multiple opponents. It would be a rare situation that someone smaller, slower, weaker, and with no ability will attack you. If this happens you certainly would not need special skills to defend against that person. This being the case you must defend from the stand point of skill, not strength.