Having trouble decrypting in C# something encrypted on iPhone using RSA

StackOverflow https://stackoverflow.com/questions/1133724

  •  16-09-2019
  •  | 
  •  

Question

I've spent two days on this so far and combed through every source at my disposal, so this is the last resort.

I have an X509 certificate whose public key I have stored in the iPhone's keychain (simulator only at this point). On the ASP.NET side, I've got the certificate in the cert store with a private key. When I encrypt a string on the iPhone and decrypt it on the server, I get a CryptographicException "Bad data." I tried the Array.Reverse suggested in the RSACryptoServiceProvider page on a longshot, but it did not help.

I have compared the base-64 strings on both sides and they're equal. I've compared the raw byte arrays after decoding and they too are equal. If I encrypt on the server using the public key, the byte array is different from the iPhone's version and readily decrypts using the private key. The raw plaintext string is 115 characters so it's within the 256-byte limitation of my 2048-bit key.

Here's the iPhone encryption method (pretty much verbatim from the CryptoExercise sample app's wrapSymmetricKey method):

+ (NSData *)encrypt:(NSString *)plainText usingKey:(SecKeyRef)key error:(NSError **)err
{
    size_t cipherBufferSize = SecKeyGetBlockSize(key);
    uint8_t *cipherBuffer = NULL;
    cipherBuffer = malloc(cipherBufferSize * sizeof(uint8_t));
    memset((void *)cipherBuffer, 0x0, cipherBufferSize);
    NSData *plainTextBytes = [plainText dataUsingEncoding:NSUTF8StringEncoding];
    OSStatus status = SecKeyEncrypt(key, kSecPaddingNone,
                                (const uint8_t *)[plainTextBytes bytes], 
                                [plainTextBytes length], cipherBuffer, 
                                &cipherBufferSize);
    if (status == noErr)
    {
        NSData *encryptedBytes = [[[NSData alloc]
                    initWithBytes:(const void *)cipherBuffer 
                    length:cipherBufferSize] autorelease];
        if (cipherBuffer)
        {
            free(cipherBuffer);
        }
        NSLog(@"Encrypted text (%d bytes): %@",
                    [encryptedBytes length], [encryptedBytes description]);
        return encryptedBytes;
    }
    else
    {
        *err = [NSError errorWithDomain:@"errorDomain" code:status userInfo:nil];
        NSLog(@"encrypt:usingKey: Error: %d", status);
        return nil;
    }
}

And here's the server-side C# decryption method:

private string Decrypt(string cipherText)
{
    if (clientCert == null)
    {
        // Get certificate
        var store = new X509Store(StoreName.My, StoreLocation.LocalMachine);
        store.Open(OpenFlags.ReadOnly);
        foreach (var certificate in store.Certificates)
        {
            if (certificate.GetNameInfo(X509NameType.SimpleName, false) == CERT)
            {
                clientCert = certificate;
                break;
            }
        }
    }

    using (var rsa = (RSACryptoServiceProvider)clientCert.PrivateKey)
    {
        try
        {
            var encryptedBytes = Convert.FromBase64String(cipherText);
            var decryptedBytes = rsa.Decrypt(encryptedBytes, false);
            var plaintext = Encoding.UTF8.GetString(decryptedBytes);
            return plaintext;
        }
        catch (CryptographicException e)
        {
            throw(new ApplicationException("Unable to decrypt payload.", e));
        }
    }
}

My suspicion was that there was some encoding problems between the platforms. I know that one is big-endian and the other is little-endian but I don't know enough to say which is which or how to overcome the difference. Mac OS X, Windows, and the iPhone are all little-endian so that's not the problem.

New theory: if you set the OAEP padding Boolean to false, it defaults to PKCS#1 1.5 padding. SecKey only has SecPadding definitions of PKCS1, PKCS1MD2, PKCS1MD5, and PKCS1SHA1. Perhaps Microsoft's PKCS#1 1.5 != Apple's PKCS1 and so the padding is affecting the binary output of the encryption. I tried using kSecPaddingPKCS1 with the fOAEP set to falseand it still didn't work. Apparently, kSecPaddingPKCS1 is equivalent to PKCS#1 1.5. Back to the drawing board on theories…

Other newly-tried theories:

  1. Certificate on iPhone (.cer file) is not exactly the same as the PKCS#12 bundle on the server (.pfx file) and so it could never work. Installed .cer file in different cert store and server-encrypted string roundtripped just fine;
  2. Conversion to base-64 and act of POSTing to server resulted in oddness that wasn't present in same class roundtrip so I first tried some URLEncoding/Decoding and then posted raw binary from iPhone, verified that it was equal, and got same bad data;
  3. My original string was 125 bytes so I thought it might be truncating in UTF-8 (long shot) so I cropped it down to a 44-byte string with no result;
  4. Looked back over the System.Cryptography library to make sure I was using an appropriate class and discovered `RSAPKCS1KeyExchangeDeformatter`, became elated at new prospects, and dejected when it behaved exactly the same.

Success!

It turned out that I had some cruft in my Keychain on the iPhone Simulator that was muddying the waters, so to speak. I deleted the Keychain DB at ~/Library/Application Support/iPhone Simulator/User/Library/Keychains/keychain-2-debug.db to cause it to be re-created and it worked fine. Thank you for all of your help. Figures it would have been something simple but non-obvious. (Two things I learned: 1) uninstalling the app from the simulator does not clear its Keychain entries and 2) start absolutely fresh periodically.)

NOTE: The generic path for the keychain file is dependent on the iOS version: ~/Library/Application Support/iPhone Simulator/[version]/Library/Keychains/keychain-2-debug.db e.g., ~/Library/Application Support/iPhone Simulator/4.3/Library/Keychains/keychain-2-debug.db

Was it helpful?

Solution

Well... the first step (as you say you have done) is to encrypt the same messages with the same initialization vectors using both the iPhone and the C# implementation. You should get the same output. You said you didn't, so there is a problem.

This means either:

  • The iPhone implementation of RSA is incorrect.
  • The .NET implementation of RSA is incorrect.
  • The key files are different (or being interpreted differently).

I would suggest the first two are unlikely, however they are remotely possible.

You state: "Installed .cer file in different cert store and server-encrypted string roundtripped just fine"... this doesn't prove anything: all this proves is that given a particular random set of numbers you can encrypt/decrypt successfully on one platform. You are not guaranteeing that both platforms are seeing the same set of random numbers.

So I suggest you take it down to the lowest level possible here. Inspect the direct (byte array) inputs and outputs of the encryption on both platforms. If with the exact same (binary) inputs you don't get the same output, then you have a platform problem. I think this is unlikely, so I'm guessing you will find that the IVs are being interpreted differently.

OTHER TIPS

this is my first answer on stackoverflow, so please forgive me if I do it wrong!

I can't give you a complete answer, however I had very similar issues when I tried to integrate with PHP - it seems that the format of Apple's certificate files is a little different from that which other software expects (including openssl).

Here's how I decrypt an encrypted signature in PHP - I actually extract the modulus and PK from the transmitted public key manually and use that for the RSA stuff, rather than trying to import the key:

// Public key format in hex (2 hex chars = 1 byte):
//30480241009b63495644db055437602b983f9a9e63d9af2540653ee91828483c7e302348760994e88097d223b048e42f561046c602405683524f00b4cd3eec7e67259c47e90203010001
//<IGNORE><--------------------------------------------- MODULUS --------------------------------------------------------------------------><??>< PK > 
// We're interested in the modulus and the public key.
// PK = Public key, probably 65537

// First, generate the sha1 of the hash string:
$sha1 = sha1($hashString,true);

// Unencode the user's public Key:
$pkstr = base64_decode($publicKey);
// Skip the <IGNORE> section:
$a = 4;
// Find the very last occurrence of \x02\x03 which seperates the modulus from the PK:
$d = strrpos($pkstr,"\x02\x03");
// If something went wrong, give up:
if ($a == false || $d == false) return false;
// Extract the modulus and public key:
$modulus = substr($pkstr,$a,($d-$a));
$pk = substr($pkstr,$d+2);

// 1) Take the $signature from the user
// 2) Decode it from base64 to binary
// 3) Convert the binary $pk and $modulus into (very large!) integers (stored in strings in PHP)
// 4) Run rsa_verify, from http://www.edsko.net/misc/rsa.php
$unencoded_signature = rsa_verify(base64_decode($signature), binary_to_number($pk), binary_to_number($modulus), "512");

//Finally, does the $sha1 we calculated match the $unencoded_signature (less any padding bytes on the end)?
return ($sha1 == substr($unencoded_signature,-20)); // SHA1 is only 20 bytes, whilst signature is longer than this.

The objective-c that generates this public key is: 

NSData * data = [[SecKeyWrapper sharedWrapper] getPublicKeyBits];
[req addValue:[data base64Encoding] forHTTPHeaderField: @"X-Public-Key"];
data = [[SecKeyWrapper sharedWrapper] getSignatureBytes:[signatureData dataUsingEncoding:NSUTF8StringEncoding]];
[req addValue:[data base64Encoding] forHTTPHeaderField: @"X-Signature"];

Using SecKeyWrapper from Apple's example project CryptoExercise (you can view the file here: https://developer.apple.com/iphone/library/samplecode/CryptoExercise/listing15.html)

I hope this helps?

Will this help you ?

Asymmetric Key Encryption w/ .NET & C#

  • Sorry for the short post, time constraints and all. Anyway, saw your Twitter request for help.. this shows how I did this with PHP and decrypted on .NET, simliar. I notice your decrypt class is slightly diff than mine, so this article might help.

I believe you've answered the question yourself. The problem most certainly lies within the endianness.

This is a possible way of writing two-way conversion methods:

short convert_short(short in)
{
 short out;
 char *p_in = (char *) &in;
 char *p_out = (char *) &out;
 p_out[0] = p_in[1];
 p_out[1] = p_in[0];  
 return out;
}

long convert_long(long in)
{
 long out;
 char *p_in = (char *) &in;
 char *p_out = (char *) &out;
 p_out[0] = p_in[3];
 p_out[1] = p_in[2];
 p_out[2] = p_in[1];
 p_out[3] = p_in[0];  
 return out;
}

This might be a good resource for you (other than wikipedia): http://betterexplained.com/articles/understanding-big-and-little-endian-byte-order/

Because you control both sides, my recommendation (if you cannot get the library encryption algorithms to work together on the two platforms) would be to write the encryption yourself on both sides, using the same algorithm.

That way you have the control, and would be able to debug the encryption internals to see what is going wrong.

It is a last resort (of course) but would probably have taken less time than the three days that you have spent already, and have a high chance of success

HTH

Licensed under: CC-BY-SA with attribution
Not affiliated with StackOverflow
scroll top