In this article, we're going to examine a comprehensive example that combines the capabilities of Crossplane and Dapr to establish and subsequently utilize a series of cloud resources. This hands-on example will clarify crucial questions in this process, such as the methods application developers use to create and tailor their personal cloud resources. Also, how do these developers explore and identify resources that they can link to directly from their application code?

‍

Crossplane, a project under the CNCF umbrella, serves as an efficient tool for provisioning cloud resources by harnessing the declarative nature of Kubernetes APIs. Upon installing Crossplane Providers (AWS, GCP, Azure, Alibaba Cloud, Helm, Kubernetes etc.), one can establish resources in multiple providers by merely identifying and managing Kubernetes Resources. Crossplane is widely recognized for its Composition capabilities (Composite Resources Definitions- XRDs), which empowers teams to formulate and manage clusters of resources (provisioned and set-up) utilizing a simplified Kubernetes Custom Resource Definition (CRD).

‍

Next, we will dive into how the Dapr project (another initiative by CNCF) equips developers with the ability to create and connect Cloud-Native applications using a set of common APIs. Dapr also presents the platform engineering team(s) with tools that simplify the creation of development environments, thus enabling them to concentrate on feature development and rectifying bugs. We will inspect how the combination of Dapr and Crossplane can reduce the cognitive burden of developers when linking and interacting with resources that might even be hosted on different cloud providers.

‍

Before we can make use of these cloud resources, we need to first create them and ensure that they're set up correctly, so application developers can access them. We will commence by creating a Crossplane Composition that allows teams to establish on-demand databases. Once we have a database for applications to retain data, we will explore linking to it by expanding the previous composition to make use of Dapr components, thus offering developers a straightforward method to identify and interact with the database instance.

Provisioning Infrastructure

Depending on your choice of cloud provider for creating resources, Crossplane requires the installation and configuration of Crossplane Providers on a Kubernetes Cluster. We will apply the Crossplane Helm Provider in this instance to establish a Redis database on the same cluster where Crossplane is located. This example can be expanded to support the creation of resources from any cloud provider of your preference.

‍

For insights on setting up and configuring your local KinD Cluster to function on your laptop, you can refer to a step-by-step tutorial at the following URL:

https://github.com/salaboy/from-monolith-to-k8s/tree/main/platform/crossplane-dapr

‍

For the sake of simplicity, let's start by creating just a Redis database for our application to connect to:

‍

Typically, regardless of whether it's a NoSQL database or a Relational Database, certain details (username/password/endpoint+port/tls enabled?) are essential to ensure your application can establish a connection once the resource is provisioned.

‍

Hold on for a moment, though; reality is seldom that simple. If you plan to provision a database, the associated configuration of this database becomes an imperative decision.

‍

Some common questions to address at this stage include:

• Do we need a highly available database?
• What's the intended location for database operation? Which global region to be precise?
• What specific version of the database do we intend to use?
• What's the designated storage capacity for the database, and where is this storage located?

‍

This stage is where a Platform Team can utilize Crossplane compositions to simplify all these intricate questions that development teams shouldn't have to contend with when all they need is a functioning database for their applications. Let's delve into how this process works.

Following the installation and configuration of Crossplane, as delineated in the step-by-step tutorial, you'll locate the Crossplane Composition within a file labeled app-database-redis.yaml:

https://github.com/salaboy/from-monolith-to-k8s/blob/main/platform/crossplane-dapr/app-database-redis.yaml

‍

The management of this Composition is handled by the subsequent CompositeResourceDefinition, which we'll employ to create new database instances: https://github.com/salaboy/from-monolith-to-k8s/blob/main/platform/crossplane-dapr/app-database-resource.yaml

‍

Upon the installation of both the Composition and CompositeResourceDefinition within the cluster, you can request a new Database instance by creating a Database resource that looks like this:

<pre class="codeWrap"><code>apiVersion: salaboy.com/v1alpha1
kind: Database
metadata:
 name: my-db
spec:
 compositionSelector:
   matchLabels:
     provider: local
     type: dev
 parameters:
    size: small
</code></pre>

‍

By creating new instances of the Database resource, teams are signaling a request for a new Database to be provisioned by the Crossplane composition. As outlined previously, the Crossplane Helm Provider will create a Redis instance using the Bitnami Redis Helm Chart. By defining a composition, the platform team can encapsulate all parameters required by the Redis Helm Chart behind a simpler resource intended for use by the application development teams (AppDev).

‍

Since Crossplane operates by expanding the Kubernetes APIs, you can utilize kubectl to enumerate all available databases.

<pre class="codeWrap"><code>kubectl get dbs
NAME    SIZE    SYNCED   READY   COMPOSITION            AGE
my-db   small   True     True    db.local.salaboy.com   3h28m
</code></pre>

‍

This composition (Database) autonomously establishes a Redis instance.

<pre class="codeWrap"><code>kubectl get pods
NAME                   READY   STATUS    RESTARTS   AGE
my-db-redis-master-0   1/1     Running   0          3h35m
</code></pre>

‍

As shown in the preceding diagram, the Redis Helm chart additionally formulates a new Kubernetes Secret encompassing the connection details needed to link to the instances we've just created.

<pre class="codeWrap"><code>kubectl get secret
NAME                          TYPE                 DATA   AGE
my-db-redis                   Opaque               1      3h37m
</code></pre>

‍

Applications can now employ this secret to establish a connection to the recently provisioned database.

‍

Ultimately, all these efforts would amount to little if we couldn't employ the same interface for provisioning resources across multiple cloud providers. This is when the labels in the composition and the Database resources become significant. For instance, if we wanted to establish an InMemoryStore (Redis) within the Google Cloud Platform, we would merely need to supply a new composition that utilizes the GCP Crossplane Provider and encapsulates all necessary details for creating that database behind the same Database resource.

‍

By formulating a new Composition and employing varying labels, our developers can repurpose the same resource to provision a database in GCP or any other Cloud Provider.

<pre class="codeWrap"><code>apiVersion: salaboy.com/v1alpha1
kind: Database
metadata:
 name: my-db-on-the-cloud
spec:
 compositionSelector:
   matchLabels:
     provider: cloud
     type: dev
 parameters:
    size: small
</code></pre>

‍

We have utilized the provider: cloud label to select the composition that establishes an InMemoryStore in GCP via the Crossplane GCP provider.

We have databases!!! So, what's the next step?

Consuming Infrastructure

As an application developer, I relish the ability to store and retrieve data from a database as and when my application demands it. However, deciphering the database's whereabouts or its specific type to pinpoint the appropriate libraries and connection details can be challenging, especially when the database may vary across environments.

‍

In this section, we'll explore Dapr, which provides a common set of APIs for interacting with infrastructure, irrespective of its location. This allows developers to concentrate on developing new features or rectifying bugs instead of grappling with boilerplate code and sophisticated setups.

‍

With Dapr, the platform team that defines where the infrastructure will be established can configure "Dapr Components" to enable developers to connect to the necessary infrastructure by simply recognizing a Dapr Component name.

‍

Now, let's observe this in action. Reverting to our sample application, you now wish to connect to the database provisioned in GCP or locally leveraging Helm.

‍

By employing Dapr, we can set up a Dapr Component that obfuscates all database details, leaving developers only needing to know the name of the Dapr Statestore Component to connect and utilize the database.

‍

The Statestore Dapr Component encompasses all the credentials and configurations required for establishing a connection with our Redis database. The application can now engage with the Statestore component employing HTTP, GRPC, or one of the Dapr SDKs, without necessitating any Redis dependency within your application code! This minimizes the number of dependencies, the size, and the attack surface of your services.

‍

Here is how the Statestore Dapr Component appears when adopting a declarative configuration method for Kubernetes:

<pre class="codeWrap"><code>apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
 name: my-statestore
spec:
 type: state.redis
 version: v1
 metadata:
 - name: redisHost
   value: <REDIS_HOST:REDIS_PORT>
 - name: redisPassword
   value: <REDIS_PASSWORD>
</code></pre>

‍

Now, how does this influence our Crossplane composition? Initially, we can allow developers to configure their Dapr components, but we can enhance their experiences on top of Kubernetes by simplifying things further.

‍

Let's include our Dapr Statestore component configuration into our Crossplane composition utilizing the Kubernetes Crossplane Provider. You can examine the composition that includes the Statestore Dapr component here: https://github.com/salaboy/from-monolith-to-k8s/blob/main/platform/crossplane-dapr/app-database-redis-dapr.yaml#L119. This new Composition is part of a separate API group (dapr.db.local.salaboy.com) and bears the label dapr-dev.

‍

Now, upon requesting a new Database resource, the Helm Provider will install the Redis Helm Chart, and the Kubernetes Provider will generate a new Kubernetes Object (as outlined by Crossplane). This creation links a Dapr Component and the credentials originating from the secret established when installing the Redis chart, as demonstrated in the figure below.

Post installation of the new composition, you have the ability to create a new Database resource adhering to the same API and schema as earlier, but this time, you'll utilise a different label to opt for the new Dapr aware composition:

<pre class="codeWrap"><code>apiVersion: salaboy.com/v1alpha1
kind: Database
metadata:
 name: my-db-dapr
spec:
 compositionSelector:
   matchLabels:
     provider: local
     type: dapr-dev
 parameters:
    size: small
</code></pre>

‍

After applying this resource, you can verify that the Dapr Statestore Component was created and connected to the Redis Instance by executing:

<pre class="codeWrap"><code>kubectl get components -n my-db-dapr
NAME                    AGE
my-db-dapr-statestore   16m
</code></pre>

‍

Finally, developers can now utilize the dapr CLI to query components within their target cluster.

<pre class="codeWrap"><code>dapr components -kNAMESPACE
 NAME                  TYPE        VERSION SCOPES CREATED  AGE
my-db-dapr my-db-dapr-statestore state.redis v1             10:17.35 13m
</code></pre>

‍

As an application developer, you are presented with two distinct yet equivalent options. You can incorporate the Dapr SDKs into your service (available for most programming languages) or employ plain HTTP/GRPC requests to interact with the Dapr Components APIs. In the step-by-step tutorial, you deploy two applications: one that stores data into Redis (Java) and another that retrieves it (Go).

‍

For example, here's how a Java application would store data in the Statestore component we just provisioned:

<pre class="codeWrap"><code>String STATE_STORE_NAME = "my-db-dapr-statestore";
private DaprClient client = new DaprClientBuilder().build();@

PostMapping("/")
public MyValues storeValues(@RequestParam("value") String value) {
      State<MyValues> results = client.getState(STATE_STORE_NAME,
                                   "values", MyValues.class).block();

      MyValues valuesList = results.getValue();

      if (valuesList == null) {
          valuesList = new MyValues(new ArrayList<String>());
          valuesList.values().add(value);
      } else {
          valuesList.values().add(value);
      }

        client.saveState(STATE_STORE_NAME, "values",
                                          valuesList).block();
      return valuesList;
}
</code></pre>

‍

This is how our Go application will access the data stored by the Java application:

<pre class="codeWrap"><code>dapr "github.com/dapr/go-sdk/client"

var (
  STATE_STORE_NAME = "my-db-dapr-statestore"
  daprClient       dapr.Client
)

func readValues(w http.ResponseWriter, r *http.Request) {
  ctx := context.Background()

  daprClient, daprErr := dapr.NewClient()
  if daprErr != nil {
      panic(daprErr)
  }

  result, err := daprClient.GetState(ctx, STATE_STORE_NAME,
                                           "values", nil)
  if err != nil {
      panic(err)
  }
  myValues := MyValues{}
  json.Unmarshal(result.Value, &myValues)

  respondWithJSON(w, http.StatusOK, myValues)
}
</code></pre>

‍

If you're not a Java or Go developer, you can explore the other available Dapr SDKs for Python, Rust, Javascript, etc.

Conclusion

In this article, we delved into how Dapr and Crossplane can be combined to provision cloud resources, which can be utilized using Dapr Components without complicating developers by requiring them to understand where these cloud resources are located or how to connect to them. Most importantly, we have empowered the Platform team(s) to express, via a simple interface, how these resources and components are collectively configured.

‍

Once you have the basics in place, you can advance to exploring other Dapr components. For example, let's say you wish to send and receive async messages between various applications. In that case, you can formulate a new Dapr Pub/Sub Component and employ Redis, Kafka, RabbitMQ, or a Cloud Provider implementation. Yet again, this places developers in a position where they can concentrate solely on constructing features without worrying about the transport used for message circulation.

‍

The latest Dapr 1.10 release contains many novel features and enhancements, such as Workflow, Pluggable Component SDKs, and Multi-App Run. You can learn all about them in this article.