What if acquiring knowledge about the Kubernetes API is all you need to set up any infrastructure? And this doesn't just apply to AWS, Azure & Google; it extends to IONOS, DigitalOcean, and even vSphere. Let's delve into Crossplane and explore how we can systematically create an S3 Bucket on AWS.No need for coding; it's entirely declarative!

‍

Crossplane asserts itself as "The cloud native control plane framework." It presents a fresh approach to managing any cloud resource, whether Kubernetes-native or not. It serves as an alternative Infrastructure-as-Code tool compared to Terraform, AWS CDK/Bicep, or Pulumi. Crossplane introduces a different abstraction level, relying on Kubernetes CRDs. Some might label it the cloud native approach to GitOps.

What Sets Crossplane Apart

Crossplane can also be compared to tools facilitating the management of cloud resources through the Kubernetes API, such as AWS Controllers for Kubernetes (ACK), Azure Service Operator for Kubernetes (ASO), or Google Config Connector. Crossplane providers are even derived from ACK and ASO.

‍

However, the CNCF incubating project Crossplane offers additional features:

The Crossplane community firmly believes that developers using Kubernetes for deploying their applications shouldn't grapple with low-level infrastructure APIs.

‍

Crossplane utilizes the Kubernetes API and extends it with a series of Custom Resource Definitions (CRDs) to abstract away from the actual cloud provider APIs. Moreover, these CRDs serve as a solid foundation for constructing an Internal Developer Platform (IDP). Crossplane advocates self-service by introducing building blocks like Composite Resource Claims (XRCs), providing an excellent API for typical application developers. Conversely, Composite Resources (XRs) and Composites are significantly more potent and ideal for platform operators.In conclusion, Crossplane also facilitates efficient interoperability across multiple clouds. And don't fret, you can even order pizza using Crossplane :).

Crossplane Fundamental Concepts

The basic principles of Crossplane revolve around the notion of Composite Resources, which are constructed from various essential components:

• Composite Resources (XR): Comprise Managed Resources to create higher-level infrastructure units, particularly beneficial for platform teams. Optional inclusion of CompositeResourceClaim (XRC), also known as a 'Claim,' serving as an abstraction for application teams.
• CompositeResourceDefinition (XRD): Defines an OpenAPI schema that the Composition must adhere to, similar to Kubernetes CRDs.
• Composition: Describes the actual infrastructure primitives (Managed Resources) used in building the Composite Resource. Multiple Compositions within one XRD, catering to different environments such as development, staging, and production.
• Nesting of Composite Resources: Allows for combining Composite Resources, facilitating higher-level abstractions and improved separation of concerns.

Example: Nested Composite Resources in an AWS EKS cluster for network/subnetting setup and EKS cluster creation.More detailed information on nested Composite Resources can be found [here].

• Packaging Composite Resource Artifacts: Optionally package Composite Resource artifacts using a Configuration into an OCI container image.
• Managed Resources (MR): Represent infrastructure primitives, mostly in cloud providers. Identified through Kubernetes custom resources (CRDs), accessible here.
• Providers: Packages bundling a set of Managed Resources and a Controller to provision infrastructure resources. Examples can be found on GitHub, such as provider-aws or on docs.crds.dev. Comprehensive list of all available Providers is also on GitHub.
• Packages: Formerly known as Stacks, these are OCI container images handling distribution, version updates, dependency management, and permissions for Providers and Configurations.

Getting Started with Crossplane: Launching a K8 Cluster with Kind

To leverage Crossplane, a Kubernetes cluster is essential for its operations. This management cluster, equipped with Crossplane, will be responsible for provisioning the specified infrastructure. Whether you opt for a managed or local Kubernetes cluster such as EKS, AKS, Minikube, or k3d, the choice is yours.

‍

In my GitHub project, I utilized kind to host the management cluster.Ulizing kind is straightforward's dive in! Ensure you have kind, the Helm package manager, and kubectl installed. On a Mac, you can use brew

<pre class="codeWrap"><code>brew install kind helm kubectl</code></pre>

Additionally, install the Crossplane CLI:c

<pre class="codeWrap"><code>curl -sL https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh | sh2
sudo mv kubectl-crossplane /usr/local/bin</code></pre>

Now, the kubectl crossplane --help command is ready for use. Now, execute the following command to spin a local knd cluster.

<pre class="codeWrap"><code>kind create cluster --image kindest/node:v1.23.0 --wait 5m</code></pre>

This command initiates the creation of a local kind cluster, waiting for 5 minutes to ensure proper setup.

Installing Crossplane with Helm

The Crossplane documentation instructs us to use Helm for the installation process.

Follow these steps:

<pre class="codeWrap"><code>1 kubectl create namespace crossplane-system
2
3helm repo add crossplane-stable https://charts.crossplane.io/stable
4helm repo update
5
6helm upgrade --install crossplane --namespace crossplane-system crossplane-stable/crossplane
</code></pre>

‍

As an alternative, powered by Renovate, we can create a simple Chart.yaml to enable automatic updates when new Crossplane versions are released.

To install Crossplane using our custom Chart.yaml, execute the following commands:

<pre class="codeWrap"><code>helm dependency update crossplane-config/install
helm upgrade --install crossplane --namespace crossplane-system crossplane-config/install
</code></pre>

Ensure exclusion of /charts directories and Chart.lock files via .gitignore. Renovate will then monitor Crossplane versions.

Before applying a Provider, ensure Crossplane is healthy and running using the kubectl wait command:

<pre class="codeWrap"><code>kubectl wait --for=condition=ready pod -l app=crossplane --namespace crossplane-system --timeout=120s</code></pre>

This step prevents errors like the following when applying a Provider:

<pre class="codeWrap"><code>error: resource mapping not found for name: "provider-aws" namespace: "" from "provider-aws.yaml": no matches for kind "Provider" in version "pkg.crossplane.io/v1"2
ensure CRDs are installed first
</code></pre>

Finally, check the Crossplane status with:

<pre class="codeWrap"><code>$ kubectl get all -n crossplane-system
NAME                                           READY   STATUS    RESTARTS   AGE
pod/crossplane-7c88c45998-d26wl                1/1     Running   0          69s
pod/crossplane-rbac-manager-8466dfb7fc-db9rb   1/1     Running   0          69s

NAME                                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/crossplane                1/1     1            1           69s
deployment.apps/crossplane-rbac-manager   1/1     1            1           69s

NAME                                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/crossplane-7c88c45998                1         1         1       69s
replicaset.apps/crossplane-rbac-manager-8466dfb7fc   1         1         1       69s
</code></pre>

Configuring Crossplane for AWS Access: Creating aws-creds.conf

Assuming you have the AWS CLI installed and configured using the aws configure command, follow the steps to create an aws-creds.conf file as outlined in the documentation:

<pre class="codeWrap"><code>echo "[default]
aws_access_key_id = $(aws configure get aws_access_key_id)
aws_secret_access_key = $(aws configure get aws_secret_access_key)
" > aws-creds.conf
</code></pre>

Caution: Never include this file in source control, as it contains sensitive AWS credentials. In the example project, I added *-creds.conf to the .gitignore file.

‍

For CI systems like GitHub Actions (utilized by this repository), ensure both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are configured as repository secrets.

Additionally, make sure the default region is configured locally or as an env variable in your CI system. For example, in GitHub Actions, all three required variables look like this:

AWS Provider Secret

To establish AWS Provider secret, follow these steps to leverage the aws-creds.conf file and generate the Crossplane AWS Provider secret:

<pre class="codeWrap"><code>kubectl create secret generic aws-creds -n crossplane-system --from-file=creds=./aws-creds.conf</code></pre>

After successful execution, a new secret named aws-creds should be available and ready for use.

‍

Crossplane AWS Provider

To enable infrastructure provisioning on a cloud platform such as AWS, the initial step is to install the appropriate Crossplane Provider. It's essential to understand that the Provider encompasses all essential Managed Resources (MRs) along with their corresponding controllers. Given that a Provider acts as a Crossplane Package, its installation can be facilitated using the Crossplane CLI as demonstrated:

<pre class="codeWrap"><code>kubectl crossplane install provider crossplane/provider-aws:v0.22.0</code></pre>

Alternatively, you have the option to craft your own provider-aws.yaml file. Note that the kind: Provider under apiVersion: pkg.crossplane.io/v1 differs significantly from the kind: Provider intended for consumption, which operates under apiVersion: meta.pkg.crossplane.io/v1.Ensure that you've installed the AWS provider utilizing the following kubectl command:

<pre class="codeWrap"><code>kubectl apply -f crossplane-config/provider-aws.yaml</code></pre>

The specified package version, coupled with the packagePullPolicy configuration, plays a pivotal role as it allows for the configuration of an upgrade strategy for the Provider. Comprehensive details regarding available fields can be accessed in the documentation. Crossplane also facilitates automated upgrades to newer versions if desired. If multiple package versions are installed, you'll observe them prefixed with providerrevision.pkg.x when executing:

<pre class="codeWrap"><code>$ kubectl get providerrevision2
NAME                                                           HEALTHY   REVISION   IMAGE                             STATE      DEP-FOUND   DEP-INSTALLED AGE3
providerrevision.pkg.crossplane.io/provider-aws-2189bc61e0bd   True      1          crossplane/provider-aws:v0.22.0   Inactive                               6d22h4
providerrevision.pkg.crossplane.io/provider-aws-d87796863f95   True      2          crossplane/provider-aws:v0.28.1   Active                                 43h</code></pre>

Now that our initial Crossplane Provider is successfully installed, verify its existence using kubectl get provider:

<pre class="codeWrap"><code>$ kubectl get provider2
NAME           INSTALLED   HEALTHY   PACKAGE                           AGE3
provider-aws   True        Unknown   crossplane/provider-aws:v0.22.0   13s
</code></pre>

Before applying a ProviderConfig to our AWS provider, it's imperative to confirm its health and operational status. This can be achieved by employing the kubectl wait command as follows:

<pre class="codeWrap"><code>kubectl wait --for=condition=healthy --timeout=120s provider/provider-aws</code></pre>

By ensuring the Provider's health, we mitigate potential errors that might arise during the subsequent application of the ProviderConfig.

ProviderConfig for AWS Credentials

To utilize the Secret containing AWS credentials, it is imperative to create a ProviderConfig object. This object instructs the AWS Provider on the location of its AWS credentials. Follow these steps to create the provider-config-aws.yaml:

‍Crossplane resources default to using the ProviderConfig named default if no specific ProviderConfig is explicitly mentioned. Therefore, this ProviderConfig will serve as the default configuration for all AWS resources.

‍

Ensure that the values for secretRef.name and secretRef.key in the ProviderConfig align with the corresponding fields in the previously created Secret.

Apply the ProviderConfig using the following command:

<pre class="codeWrap"><code>kubectl apply -f crossplane-config/provider-config-aws.yaml</code></pre>

Set Up S3 Bucket Provisioning with Crossplane

The core controller of Crossplane along with the AWS Provider controller is now prepared to provision various infrastructure components in AWS. Let's initiate the process with a straightforward S3 Bucket creation.To employ Composite Resources, the initial step involves configuring Crossplane.

‍

This ensures that Crossplane is aware of the desired Composite Resources (XRs) and knows how to handle them when created. This configuration is achieved through a CompositeResourceDefinition (XRD) resource and one or more Composition resources.

‍

To provision an S3 Bucket in AWS using Crossplane, you need to assemble three fundamental building blocks:

• CompositeResourceDefinition (XRD)
• Composition
• Composite  (XR) or Claim (XRC)

CompositeResourceDefinition (XRD) for the S3 Bucket

tAll potential components of an XRD are detailed in this documentation. The field spec.versions.schema should incorporate an OpenAPI schema, akin to those employed by Kubernetes CRDs. This schema dictates the fields that the XR (and Claim) will possess. Comprehensive CRD documentation and a tutorial on crafting OpenAPI schemas can be found in the Kubernetes docs.It's important to note that Crossplane will automatically augment this section.

‍

This includes the following fields, which will be disregarded if present in the schema:

• spec.resourceRef
• spec.resourceRefs
• spec.claimRef
• spec.writeConnectionSecretToRef
• status.condition
• status.connectionDetails

The GitHub example project hosts a Composite Resource Definition (XRD) for an S3 Bucket. The definition.yaml might resemble the following:

‍

The file employs numerous comments to assist in creating your own XRD. A CompositeResourceDefinition can be broadly divided into the top area featuring Crossplane-specific configuration (with spec.group, spec.names, spec.claimNames, and a compositionRef using spec.defaultCompositionRef) and the bottom section utilizing the OpenAPI schema to define the parameters of our resources. In our case, where we aim to provision an S3 Bucket, there are only two parameters: bucketName and region.

‍

To incorporate the XRD into our cluster, execute the following command:

<pre class="codeWrap"><code>kubectl apply -f aws/s3/definition.yaml</code></pre>

‍

Optionally, one can verify the CRDs being created with the command kubectl get crds and and filter them using grep to our group name crossplane.jonashackt.io:

<pre class="codeWrap"><code>$ kubectl get crds | grep crossplane.jonashackt.io2
objectstorages.crossplane.jonashackt.io                         2022-06-27T09:54:18Z3
xobjectstorages.crossplane.jonashackt.io                        2022-06-27T09:54:18Z
</code></pre>

Crafting a Composition to Provision an S3 Bucket for Static Website Hosting

The primary task in Crossplane involves creating Compositions. This is crucial as they interact with the foundational elements provided by cloud provider APIs. Detailed documentation is available for various manifest configurations. An example of a Composition to manage an S3 Bucket in AWS, allowing public access for static website hosting, can be found in the project's composition.yaml:

‍

This example, annotated with comments, aims to guide you in creating your own Composition. After defining metadata labels, spec.compositeTypeRef, and spec.writeConnectionSecretsToNamespace, the actual resource configuration takes place in spec.resources.

As we intend to provision an S3 Bucket in AWS, consulting the Crossplane AWS provider API docs is recommended. Additionally, referencing the Terraform docs or searching the internet for example code can be beneficial.

‍

To install the Composition, use the following command: kubectl apply -f aws/s3/composition.yaml3.

Crafting a Composite Resource (XR) or Claim (XRC)

In Crossplane, only the platform team typically possesses the permissions to directly create XRs. Others use a lightweight proxy resource known as CompositeResourceClaim (XRC or "Claim") to create them with Crossplane. If you're familiar with Terraform, you can think of an XRD as similar to variable blocks of a Terraform module. The Composition could then be seen as the remaining HCL code describing how to use those variables to create actual resources. Regardless of your role, you only need to write a Composite Resource (XR) or Claim (XRC)! Crafting both is unnecessary, as the XR will be automatically generated from the XRC by Crossplane.

‍

If you take on the role of a platform engineer, you can begin crafting the XR directly, and no Claim will be generated.As we aim to create an S3 Bucket, the example project provides a claim.yaml:

‍

One of Crossplane's more basic building components is the Claim. Ensure the correct apiVersion and name the kind exactly as defined in the CompositeResourceDefinition (XRD). A namespace is required to define a valid Claim (which is not true for XRs, as they are cluster-scoped). Additionally, a compositionRef or alternatively compositionSelector must be defined to reference the Composition the Claim should use. Finally, both parameters, bucketName, and region must be defined.

‍

Applying our Claim using the command kubectl apply -f aws/s3/claim.yaml will trigger the provisioning of our S3 Bucket in AWS.The CLI validation will assist you in debugging your configuration, providing hints for any remaining problems.

<pre class="codeWrap"><code>$ kubectl apply -f aws/s3/claim.yaml2
error: error validating "claim.yaml": error validating data: [ValidationError(S3Bucket.metadata): unknown field "crossplane.io/external-name" in io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta_v2, ValidationError(S3Bucket.spec): unknown field "parameters" in io.jonashackt.crossplane.v1alpha1.S3Bucket.spec, ValidationError(S3Bucket.spec.writeConnectionSecretToRef): missing required field "namespace" in io.jonashackt.crossplane.v1alpha1.S3Bucket.spec.writeConnectionSecretToRef, ValidationError(S3Bucket.spec): missing required field "bucketName" in io.jonashackt.crossplane.v1alpha1.S3Bucket.spec, ValidationError(S3Bucket.spec): missing required field "region" in io.jonashackt.crossplane.v1alpha1.S3Bucket.spec]; if you choose to ignore these errors, turn validation off with --validate=false</code></pre>

Waiting for Resources to Become Ready

After executing kubectl apply -f aws/s3/claim.yaml, you may need to check several aspects while your resources are being deployed. The most comprehensive overview can be obtained with kubectl get crossplane, which lists all Crossplane resources:

<pre class="codeWrap"><code>$ kubectl get crossplane
NAME                                                                                          ESTABLISHED   OFFERED AGE
compositeresourcedefinition.apiextensions.crossplane.io/xs3buckets.crossplane.jonashackt.io   True          True      23m

NAME                                               AGE
composition.apiextensions.crossplane.io/s3bucket   2d17h7
NAME                                      INSTALLED   HEALTHY   PACKAGE                           AGE
provider.pkg.crossplane.io/provider-aws   True        True      crossplane/provider-aws:v0.22.0   4d21h10

NAME                                                           HEALTHY   REVISION   IMAGE                             STATE    DEP-FOUND   DEP-INSTALLED   AGE
providerrevision.pkg.crossplane.io/provider-aws-2189bc61e0bd   True      1          crossplane/provider-aws:v0.22.0   Active                               4d21h

NAME                                        AGE     TYPE         DEFAULT-SCOPE
storeconfig.secrets.crossplane.io/default   5d23h   Kubernetes   crossplane-system
</code></pre>

‍

Additional useful commands include:

• kubectl get claim: Retrieves all resources of all Claim kinds, such as PostgreSQLInstance.
• kubectl get composite: Fetches all resources of kind Composite (XR), such as XPostgreSQLInstance.
• kubectl get composition: Provides a list of all Compositions.
• kubectl get managed: Shows all resources representing a unit of external infrastructure.
• kubectl get name-of-provider: Lists all resources related to the Provider.

Troubleshooting Crossplane Configuration

If you encounter issues with your Crossplane configuration, you can efficiently identify and resolve errors by following these steps, as explained in the documentation:

• Locating Errors: According to Kubernetes conventions, Crossplane keeps errors close to their origin. If your Claim is not getting ready due to problems with your Composition or a composed resource, you need to "follow the references" to pinpoint the issue. The Claim will only indicate that the XR is not yet ready.
• Following the References:To understand and address the issue, follow these steps:a. Find your XR by executing kubectl describe claim-kind claim-metadata.name and locate its "Resource Ref" (also known as spec.resourceRef). Run kubectl describe on your XR to identify any issues with the Composition you are using, if applicable. If there are no apparent issues but your XR is not becoming ready, check for "Resource Refs" (or spec.resourceRefs) to identify your composed resources. Run kubectl describe on each referenced composed resource to determine its readiness and identify any encountered issues.

Examine the S3 Bucket and Set up a Static Website

Now, let's verify our Claim using the following kubectl command ie; kubectl get claim-kind claim-metadata.name

<pre class="codeWrap"><code>$ kubectl get ObjectStorage managed-s32
NAME         READY   CONNECTION-SECRET               AGE3
managed-s3           managed-s3-connection-details   5s
</code></pre>

‍

To monitor the readiness of the provisioned resources, execute the following kubectl command ie; kubectl get crossplane -l crossplane.io/claim-name=claim-metadata.name:

<pre class="codeWrap"><code>kubectl get crossplane -l crossplane.io/claim-name=managed-s3</code></pre>

‍

Additionally, you can confirm the successful creation of the S3 Bucket using the aws CLI with the following command aws s3 ls:

<pre class="codeWrap"><code>$ aws s3 ls
2022-06-27 11:56:26 microservice-ui-nuxt-js-static-bucket
...
</code></pre>

‍

At this point, the bucket should be provisioned and visible in the AWS console.

‍

To thoroughly confirm its functionality, let's deploy a website (the sample project includes a basic index.html) to our S3 Bucket using the aws CLI as demonstrated below:

<pre class="codeWrap"><code>aws s3 sync static s3://microservice-ui-nuxt-js-static-bucket --acl public-read</code></pre>

‍

After executing the command, we can access our deployed website by opening the following URL in our browser:http://microservice-ui-nuxt-js-static-bucket.s3-website.eu-central-1.amazonaws.com

‍Upon visiting the URL, we should observe our website already successfully deployed.

To delete the S3 Bucket, we need to remove the Claim. However, before deleting the Claim, it's essential to delete our index.html to avoid encountering errors such as "BucketNotEmpty: The bucket you tried to delete is not empty."

Execute the following command to remove the index.html file:

<pre class="codeWrap"><code>aws s3 rm s3://microservice-ui-nuxt-js-static-bucket/index.html </code></pre>

‍

Now, we can proceed to remove our S3 Bucket by using the kubectl command:

<pre class="codeWrap"><code>kubectl delete claim managed-s3</code></pre>

For a comprehensive demonstration of all the steps and commands described in this post, refer to the example repository's GitHub Actions workflow provision.yml. Incorporating these actions into a CI/CD pipeline is a best practice, ensuring that all processes are automatically executable and easily understandable!

Master the Kubernetes API with Crossplane

Embrace the power of Crossplane to dominate the world of infrastructure. It's that straightforward: Crossplane extends the Kubernetes API, eliminating the need for prolonged stays in your YAML manifests. However, there's a bit of groundwork required. Setting up a management cluster is a prerequisite for unleashing the full potential of Crossplane. You can achieve this seamlessly using GitHub Actions CI/CD workflows coupled with kind or k3d.

‍

Crossplane Providers offer extensive coverage across numerous infrastructure providers. With the introduction of Terrajet, a Terraform-to-Crossplane CRD generator, Crossplane now mirrors the capabilities of Terraform. Whatever Terraform can provision, Crossplane can handle as well.

‍

The real magic lies within Compositions! Crafting them requires skilled platform engineers who use Managed Resources as building blocks. Proficiency is key, especially when dealing with intricate infrastructure setups like an AWS EKS cluster. While there are guides on creating a "production-ready EKS cluster with Crossplane," I prefer not to label them as such.Upon my first encounter with Crossplane, it seemed like there might be something lacking—a curated library of higher-level abstractions, like what we find in the Pulumi Crosswalk collection, for instance.

‍

Although I came across some initiatives from cloud vendors like AWS Blueprints for Crossplane, I sensed a gap. A deeper investigation led me to Upbound, the driving force behind Crossplane. This company boasts a team of highly skilled individuals, including Nate Reid, a Staff Solutions Engineer at Upbound. I highly recommend checking out Nate Reid's blog for valuable insights.

‍

Additionally, Upbound is working on the Upbound Platform Reference Architectures, which appear quite promising. These architectures might just be the Composite Resource library that I initially felt was missing. I'm eager to delve deeper into them in a future post!To stay abreast of Crossplane developments, it's advisable to regularly check the roadmap and visit both the Crossplane and Upbound blogs. Keeping a watchful eye on these sources can provide valuable updates and insights.