Generally, one should ask for oral argument if the appellate relief one is asking for is out-of-the-ordinary, that is, if it requires the court to take action it would rarely take. For example, appellants asking the appellate court to overrule a trial judgment should generally ask for oral argument. Why? Because appellate courts rarely overrule trial judgments, and appellant's counsel should try to get the case away from being solely considered by staff attorneys and law clerks and bring the case to an appellate justice's attention. A justice is much more likely to lead the court to do something unusual than a lowly court employee. In this example, appellees should generally try to avoid oral argument. To show how far some appellees take this avoidance, for a long time, appellate prosecutors in Houston would not show up for oral argument where the State was an appellee.
A football coach, famous for letting his quarterback run rather than pass observed that when a player passed the ball, there were three possible results (incompletion, interception, completion), and two of them were bad (incompletion and interception). Appellees' counsel feel the same way; oral argument may lose a case for them, but it is unlikely to win one for them.
Similarly, counsel asking for extraordinary relief such as mandamus or habeas corpus relief should ask for oral argument to explain why their particular case is the one for which unusual action should be taken.