EAP-pwd: Fix KDF for group 21

The previous EAP-pwd KDF implemented has an issue with group 21, that is
an elliptic curve group based on a 521 bit prime. 521 is not an even
multiple of 8, and therein lies the problem.

OpenSSL's BN library interprets a string of bits as in big-endian format
so all the calls of BN_bin2bn() will take the binary blob of bits and
turn it into a big number in big-endian format. In the EAP-pwd KDF, I am
stretching the key to "primebitlen". When that is not an even multiple
of 8 I have to mask off the excess. But I was masking off the excess
bits in the 1st octet (big endian after all) but that isn't right. The
KDF produces a string of endian-less bits. The 521st bit is the first
bit in the last octet, not the 7th bit in the first octet. So that has
been fixed and you can see in the attached diff what I'm doing.

1 files changed, 14 insertions, 2 deletions

diff --git a/src/eap_common/eap_pwd_common.c b/src/eap_common/eap_pwd_common.c
index c24b146..5836cba 100644
--- a/src/eap_common/eap_pwd_common.c
+++ b/src/eap_common/eap_pwd_common.c
@@ -75,8 +75,8 @@ void eap_pwd_kdf(u8 *key, int keylen, u8 *label, int labellen,
 
 	/* since we're expanding to a bit length, mask off the excess */
 	if (resultbitlen % 8) {
-		mask >>= ((resultbytelen * 8) - resultbitlen);
-		result[0] &= mask;
+		mask <<= (8 - (resultbitlen % 8));
+		result[resultbytelen - 1] &= mask;
 	}
 }
 
@@ -189,6 +189,18 @@ int compute_password_element(EAP_PWD_group *grp, u16 num,
 			    prfbuf, primebitlen);
 
 		BN_bin2bn(prfbuf, primebytelen, x_candidate);
+
+		/*
+		 * eap_pwd_kdf() returns a string of bits 0..primebitlen but
+		 * BN_bin2bn will treat that string of bits as a big endian
+		 * number. If the primebitlen is not an even multiple of 8
+		 * then excessive bits-- those _after_ primebitlen-- so now
+		 * we have to shift right the amount we masked off.
+		 */
+		if (primebitlen % 8)
+			BN_rshift(x_candidate, x_candidate,
+				  (8 - (primebitlen % 8)));
+
 		if (BN_ucmp(x_candidate, grp->prime) >= 0)
 			continue;