Dataform "Illegal base64 character 2d" for Bitbucket SSH connection

Google Cloud Data Analytics
,
1

I am attempting to connect a Dataform repository to a private Bitbucket repository using SSH authentication. The connection consistently fails with the error message “Illegal base64 character 2d” when trying to link the repository in Dataform settings, and the same error occurs when attempting to create a workspace within the Dataform repository. This indicates a fundamental issue with how the SSH private key is being parsed.

Troubleshooting Steps Performed:

  1. SSH Key Generation:

    • Initially generated an ED25519 SSH key pair (ssh-keygen -t ed25519).
    • Later, generated a new RSA 4096-bit SSH key pair (ssh-keygen -t rsa -b 4096).
    • In both cases, confirmed the private keys were generated without a passphrase (-N “”).

  2. SSH Key Storage in Google Cloud Secret Manager:

    • For both ED25519 and RSA keys, the private key file (without .pub extension) was stored in Google Cloud Secret Manager.
    • The file was uploaded directly using the “Upload file” option in Secret Manager to avoid copy-paste formatting issues.
    • Confirmed the Dataform service account ([email protected]) has the roles/secretmanager.secretAccessor role on the secret containing the private key.

  3. Bitbucket SSH Key Setup:

    • The public key (the .pub file content) from the generated key pair was added to the Bitbucket user account’s SSH keys (under Personal Settings → SSH keys). This was done to ensure write access, as Bitbucket’s repository “Access keys” (deploy keys) are read-only.
    • Confirmed the Bitbucket user account has “Write” access to the specific repository.

  4. Dataform Git Connection Configuration (SSH):

    • Remote Repository URL: [email protected]:.git
    • Authentication Method: Set to “SSH”.
    • Secret: Selected the Secret Manager secret containing the relevant private key.
    • SSH Public Host Key Value: Provided the full Bitbucket host key (e.g., bitbucket.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIazEu89wgQZ4bqs3d63QSMzYASSAVa0MuJ2e2gKTKqu+UUO or the RSA equivalent for consistency).

  5. Local SSH Test (Crucial Diagnostic):

    • Example: "Ran ssh -T -i ~/.ssh/dataform_bitbucket_id_rsa_FINAL [email protected] locally, and it successfully authenticated with ‘authenticated via ssh key.’ This confirms the key pair and Bitbucket setup are functional outside of GCP.

Current Status:

Despite extensive troubleshooting, including trying different SSH key types, meticulous steps for key generation and storage, and even attempting HTTPS authentication, the primary issue of the SSH private key failing to parse with “Illegal base64 character 2d” persists when configuring the Dataform repository and creating workspaces. This indicates a very low-level problem in how Google Cloud (Secret Manager/Dataform) is interpreting the private key content from Secrets Manager. Any help would be appreciated!

2

The issue “Illegal base64 character 2d” likely shows that there is a mistake with how the SSH private key is being handled, most likely because of how the key is taken from the Google Cloud Secret Manager and the formatting expected by Dataform.

To solve the issue at hand, I suggest these specific steps:

Check If the Right Private Key Format is Used

Usually, Dataform along with the related git client expects the private key to be in OpenSSH PEM format and expects certain metadata, line breaks, and extra information to be missing. It is possible some newer OpenSSH private keys (starting with -----BEGIN OPENSSH PRIVATE KEY-----) do not work well. When dealing with this type of key, my advice is to convert it to the older PEM format:

ssh-keygen -p -m PEM -f ~/.ssh/id_rsa

This command is useful as it changes the private key to the “PEM format (-----BEGIN RSA PRIVATE KEY-----)” which is reliably supported.

Do Not Use File Upload in the Secret Manager UI

Using the “Upload file” feature through the Secret Manager UI automatically changes line endings and adjusts hidden characters. This behavior affects the private key when this method is used. Instead, I suggest splitting the file and adding the private key as plain text secret:

cat ~/.ssh/id_rsa | gcloud secrets versions add YOUR_SECRET_NAME --data-file=-

This keeps the line breaks exact as it captures the secret without adding any extra or hidden marks.

Check for Hidden or Invalid Characters
Check the private key file to make sure there are no extra whitespace characters, line endings (\r), or invalid base64 characters (the dash ‘-’ is ASCII 0x2d which may show up if line breaks are read wrong). Use:

cat -A ~/.ssh/id_rsa

to check unwanted characters.

Confirm Secret Access Control
Make sure the service account of Dataform has permission for the secret and check that no changes (e.g., automatic base64 encoding /decoding) happen on retrieval.

Consider Using SSH with App Passwords as a Temporary Step
Because HTTPS authentication works, it would make sense to switch to using Bitbucket App Passwords via HTTPS while fixing SSH as a temporary workaround.

Check Dataform and GCP Documentation/Support
This usually happens because of a small logic error inside the combination of Dataform and Google Cloud SDK, especially how secrets are retrieved and SSH keys are read. Look into recent updates, known problems, or maybe send a support case with Dataform or GCP.

3

Hi @tanmay-parkable ,

Welcome to Google Cloud Community!

The “Illegal base64 character 2d” error you’re encountering when connecting Dataform to a private Bitbucket repository via SSH almost certainly points to an issue with the format of the private key stored in Google Cloud Secret Manager. The error indicates that the key is not in the expected PEM format, which uses Base64 encoding.

The most likely cause of this error is that the private key stored in Secret Manager includes the -----BEGIN OPENSSH PRIVATE KEY----- and -----END OPENSSH PRIVATE KEY----- headers and footers. The hyphen (-), which is character 2d in hexadecimal, is not a valid Base64 character, leading to the parsing failure.

To resolve this, you need to store only the Base64 encoded portion of the private key in Secret Manager.

1. Extract the Base64 Content:

  • Open your private key file (e.g., id_rsa or id_ed25519) in a text editor. Copy only the block of text between the -----BEGIN…----- and -----END…----- lines. This is the Base64 encoded key.

2. Update the Secret in Secret Manager:

  • Navigate to your secret in the Google Cloud Console.
  • Create a new version of the secret.
  • Paste the copied Base64 content as the new secret value. Ensure there are no leading or trailing spaces or newlines.
  • Disable all previous versions of the secret to ensure Dataform uses the new, correctly formatted version.

3. Re-link the Repository in Dataform:

  • Go back to your Dataform repository settings. Attempt to link the Bitbucket repository again using the same SSH configuration, pointing to the updated secret. The connection should now succeed.

If none of these solutions work, you might need to re-upload the private key using a different method or reach out to Google Cloud Support. When reaching out, include detailed information and relevant screenshots of the errors you’ve encountered. This will assist them in diagnosing and resolving your issue more efficiently.

Was this helpful? If so, please accept this answer as “Solution”. If you need additional assistance, reply here within 2 business days and I’ll be happy to help.

4

I have the same error. Neither of the replies were a solution. Google needs to improve the public / private certificate handling in dataform to make this usable. Unfortunately, this is a standards mess – many pub/private certificate file formats that are just strings without clear identification that make it hard to know what format wil be created, and what will be accepted. The best thing for google to do is show examples of formats that will be accepted in their documentation, and then have the code produce much clearer error messages and guidance if anything goes wrong.