Quick introduction to RSA with OpenSSL

Working on a project, I have encountered the need to use OpenSSL for certain operations related to RSA, that I hadn’t had the chance to try until last week.

I leave here a little of my experience and summary of use, both in the command line and in the use of the programming library.

Note: I have used it only in Windows environment, so I am not sure if the syntax is exactly the same.

Generate the public and private key pair

First we generate the private key in PEM format (the number at the end is the size of the key, if omitted it will be 512 bits):

openssl genrsa -out private.pem 1024

Now, we extract the public key:

openssl rsa -pubout -in private.pem -out public.pem

Sign files

For the project I mentioned, I needed to verify the authenticity of certain files. For this I decided to use a fairly traditional verification protocol consisting of signing the files with the private key and, upon receiving the files, decrypt them with the public key and verify that the content matches the unsigned file. Both steps are necessary (decrypt and check) to avoid that correctly signed versions, but not the desired ones, can be sent instead of the correct file.

To sign a file:

openssl rsautl -sign -in plain.txt -out signed.txt -inkey private.pem

We can verify ourselves that the signed file is correct using:

openssl rsautl -verify -in signed.txt -inkey public.pem -pubin

The contents of the file should be displayed. In case the file was not signed with the OpenSSL private key it will give an error.

Verify the signed file using C/C++

This is where I had more problems and what led me to write this post, so that others can take advantage of my blows to the air. I will make as few assumptions as possible so that everything is clear. I will only assume that the header file paths and all that is configured.

Required OpenSSL header files:

#include <openssl/err.h>
#include <openssl/bio.h>
#include <openssl/pem.h>
#include <openssl/evp.h>
#include <openssl/rsa.h>

When loading the public key, I consider two options (although I am in favor of the second):

  1. From a file. The problem is that they can easily change the public key to another and falsify the signed files. The code would be:
    BIO* pubkeyin = BIO_new(BIO_s_file());
    if (BIO_read_filename (pubkeyin, "public.pem") <= 0)
     // Error reading public key
  2. From a string of characters. This option is more generic and allows, in addition to loading the string from a file, embedding it in the executable or downloading it from a server, for example.
    BIO* pubkeyin = BIO_new_mem_buf(_public_key, strlen (public_key_str));

    Where `public_key_str` is a character string of type `char[]`.

    Important note if the character string is embedded, and is to include line breaks when copying the key to the source code.

    char public_key_str [] = "----- BEGIN PUBLIC KEY -----\n"
    "6kdxeIL8V/yNSN6LJB84 + LmVLjdP9or/hZ/l/XyAc9LT8q0Dl6p8mNzOoTe7e6CJ\n"
    "----- END PUBLIC KEY -----\n";

Extract the public key and prepare the necessary data structures:

EVP_PKEY* pkey = PEM_read_bio_PUBKEY(pubkeyin, NULL, NULL, NULL);
RSA* rsa = EVP_PKEY_get1_RSA(pkey);

The content of the signed file must be in a string of unsigned char[] type:

unsigned char* rsa_in = (unsigned char *)OPENSSL_malloc(keysize* 2);
BIO* in = BIO_new_file(_targetURL.toLocal8Bit(). Data(), "rb");
int rsa_inlen = BIO_read(in, rsa_in, keysize* 2);

Finally, decrypt the signed file (RSA_PKCS1_PADDING is the standard padding value when signing files):

unsigned char* rsa_out = (unsigned char *)OPENSSL_malloc(keysize + 1);
int rsa_outlen = RSA_public_decrypt(rsa_inlen, rsa_in, rsa_out, rsa, RSA_PKCS1_PADDING);
if (rsa_outlen <= 0) {
 // Error, file signed incorrectly
 // Here your error code
} else {
 // Important convert the read string to a character string valid for C
 rsa_out [rsa_outlen] = 0;

To check the validity of the file, it must be checked against the expected content (in this case stored in the valid_str variable):

const bool valid = (strcmp((char *)rsa_out, valid_str) == 0);