There are two different equally useful ways to think about how to distinguish rational from non-rational argumentation.
The low bar version (my preferred one) is that people engaged in a disagreement:
1) can articulate the conditions under which they would change their mind, and those conditions are plausible;
2) have internally consistent arguments (this amounts to their not contradicting their own arguments, and it’s generally about consistent major premises);
3) hold one another to the same standards in terms of logic, evidence, tone (so that if I appeal to personal experience, then I have to treat my interlocutor’s appeal to personal experience as equally valid).
A higher bar is the one set by the pragma-dialectical school. Their ten rules are set out in Argumentation, Communication, and Fallacies.
1) Freedom rule
Parties must not prevent each other from advancing standpoints or from casting doubt on standpoints.
2) Burden of proof rule
A party that advances a standpoint is obliged to defend it if asked by the other party to do so.
3) Standpoint rule
A party’s attack on a standpoint must relate to the standpoint that has indeed been advanced by the other party.
4) Relevance rule
A party may defend a standpoint only by advancing argumentation relating to that standpoint.
5) Unexpressed premise rule
A party may not deny premise that he or she has left implicit or falsely present something as a premise that has been left unexpressed by the other party.
6) Starting point rule
A party may not falsely present a premise as an accepted starting point nor deny a premise representing an accepted starting point.
7) Argument scheme rule
A party may not regard a standpoint as conclusively defended if the defense does not take place by means of an appropriate argumentation scheme that is correctly applied.
8) Validity rule
A party may only use arguments in its argumentation that are logically valid or capable of being made logically valid by making explicit one or more unexpressed premises.
9) Closure rule
A failed defense of a standpoint must result in the party that put forward the standpoint retracting it and a conclusive defense of the standpoint must result in the other party retracting its doubt about the standpoint.
10) Usage rule
A party must not use formulations that are insufficiently clear or confusingly ambiguous and a party must interpret the other party’s formulations as carefully and accurately as possible.