The bignum code has two representations of zero, since

bn_restore_invariant (and the many loops that duplicate it) leaves a
single zero word in a bignum representing 0, whereas the constant
'Zero' does not have any data words at all. Cope with this in
bignum_cmp.

(It would be a better plan to decide on one representation and stick
with it, but this is the less disruptive fix for the moment.)

[originally from svn r9996]

diff --git a/sshbn.c b/sshbn.c
index cbd710d5aea4bd68d39a103274166b26af963241..1c5c46a1eb8746862e6b5cf45e63cbaa3b9c5ced 100644 (file)
--- a/sshbn.c
+++ b/sshbn.c
@@ -1322,6 +1322,12 @@ int bignum_cmp(Bignum a, Bignum b)
     int amax = a[0], bmax = b[0];
     int i;
 
+    /* Annoyingly we have two representations of zero */
+    if (amax == 1 && a[amax] == 0)
+        amax = 0;
+    if (bmax == 1 && b[bmax] == 0)
+        bmax = 0;
+
     assert(amax == 0 || a[amax] != 0);
     assert(bmax == 0 || b[bmax] != 0);