Bloggo, il blog di Buildo
Security & DevOps

Building Value for Developers With an Internal Developer Platform

This blog post reveals how Buildo's DevOps team boosts developer autonomy by removing barriers to launching new projects and showcasing their powerful internal developer platform and essential tools.

Sandro Taje
Software Engineer
June 13, 2024
7
minutes read

At Buildo, we consistently begin every project with an understanding of the needs implied, including internal projects. As part of the DevOps team, we decided to take action to improve our developer team's workflow and autonomy (you can read how we started the development of our Clustero Developer Platform in our previous blog post, "The Missing Link in Simplifying Database Complexity”). To quickly sum up, we identified the key features that a developer would require through interviews and previous experience:

  • Expose the app to the internet via an HTTPS URL
  • Authentication for access segregation utilizing Buildo’s Google User
  • The capability of injecting secrets from 1Password, which we use commonly at Buildo

Currently, developers wanting to explore new technology need the DevOps team to start. Our objective is to remove barriers that inhibit independent experimentation. Unchecked, this leads to an overabundance of unique, snowflake environments that are difficult to manage and share knowledge.

In this post, we will discuss our approach to enabling independent experimentation without DevOps support, and the implementation details of our solutions. We'll also cover how developers can interface with our system using simple YAML files, the role of Argo CD and the App Operator, and our plans for future developments to integrate databases seamlessly.

Interface with Developers

So, how can a developer experiment without generating snowflakes or requiring DevOps support?

The answer is as simple as adding a rudimentary YAML file to a repository, allowing ArgoCD to seamlessly install your app into the cluster.

Argo CD, a declarative, GitOps continuous delivery tool for Kubernetes, will automatically examine Buildo’s repositories for the presence of an App CR (the Custom Resource defined by the Operator) and subsequently install it into the cluster, triggering the App operator for the primary installation.

apiVersion: applications.buildo.io/v1alpha1
kind: App
metadata:
  name: my-app
  namespace: my-app
spec:
  image: my-app-image:v1.2.3

Simple, right? But how about acquiring an internet address without divulging your work to the world? A few extra lines and the app operator will manage this on your behalf.

apiVersion: applications.buildo.io/v1alpha1
kind: App
metadata:
  name: my-app
  namespace: my-app
spec:
  image: my-app-image:v1.2.3
  env:
    MY_ENV_VAR: wow
  route:
    domain: my-app.buildo.io
    port: 3000
    path: /custom-path
  auth:
    enabled: true
  secretsFrom: 1password-item
  storage:
    - /cache
    - /data

The App CR in Detail

You may have noticed that the App kind isn't a native K8s resource. We've extended the Kubernetes API with a custom resource definition to allow an operator, Buildo’s App operator, to convert this simple YAML into multiple, complex resources.

The App Operator is a crucial part of our architecture. It was created using the Operator SDK with Helm. We defined the Custom Resource Definition as our agreement with developers. We chose Helm as a templating language because it is widely used and easy to understand. However, one downside of Helm is that the client controls the package content and version. With the Helm Operator approach, the DevOps team is responsible for maintaining the operator. This means that developers don't have to change the package version if an infrastructure component is updated. You can also check out our previous blog post on this topic for more information.

The App Operator reads the App kind, and converts it into different CRs and native resources.

Ingress, Service, and Certificate

In the Ingress, we link the user-specified domain to the Service. The domain links to the load balancer's IP via ExternalDNS. If the developer opts for authentication, the operator includes the required annotations to incorporate Ory Oathkeeper. An affiliated Certificate - issued by the cert-manager operator using Let’s Encrypt - is generated automatically.

...
spec:  
  route:
    domain: my-app.buildo.io
    port: 3000
    path: /custom-path
  auth:
    enabled: true
 ...

Ory Oathkeeper authorizes incoming HTTP requests. It can function as a Policy Enforcement Point within your cloud architecture, serving as a reverse proxy for your upstream API or web server, rejecting unauthorized requests and forwarding authorized ones.

ExternalSecret

ExternalSecret is essential to generate a Secret from a 1Password item, which is then bound as an environment variable in the Deployment. ExternalSecret's operator and 1Password Secrets Automation facilitate this.

...
spec:  
	secretsFrom: 1password-item
...

External Secrets Operator is a Kubernetes operator that integrates external secret management systems like AWS Secrets Manager, HashiCorp Vault, Google Secrets Manager, Azure Key Vault, IBM Cloud Secrets Manager, CyberArk Conjur, and more. The operator fetches information from external APIs and automatically injects the data into a Kubernetes Secret.

PersistenceVolumeClaim

The PersistenceVolumeClaim (PVC) procures a volume that is automatically mounted to our Deployment. The PVC creates a PersistenceVolume using the default storage class of clustero: EBS through the aws-ebs-csi-driver.

...
spec:  
	storage:
    - /var/lib/cache
    - /data
...

Our First Case Study: Conto and the Evolution of our Developer Platform

We didn't have to wait long for the first opportunity to test our work: Conto, one of our internal tools, needed something like our platform to improve its workflow. Since it was an internal tool without a specific budget, it was quite simple: a Python web app with Streamlit. After some time, together with the Conto’s team, we realized they were using a volume to cache some data, and they wanted not to lose it every time the app was restarted for an update. We saw an opportunity to enhance our Clustero Developer Platform by addressing Conto's problem, so we decided to augment the platform's features, adding the possibility of persisting volumes and ensuring that data would not be lost during app restarts.

This success story quickly resonated with other internal teams, leading to the adoption of Clustero new feature across various small projects. What started as a simple intuition evolved into a feature that empowered multiple teams and reduced dependencies on DevOps support.

Future Developments

We've succeeded in eliminating snowflake environments in favor of rapidly deployed applications. Our journey does not end here, though. We aim to integrate our solution into preexisting projects. To achieve this, we need a crucial new feature: databases. As you may have read in Benedetto’s post, in Clustero, we are already providing a straightforward, declarative method to operate a database and will incorporate this feature into the App CR.

Sandro Taje
Software Engineer

I transitioned from a full-stack software engineer to focusing on DevOps. I'm passionate about using Vim and k9s in the terminal and have diverse experience in software development and optimizing DevOps processes. Driven by my enthusiasm for technology, I'm always seeking new challenges and innovative solutions to enhance project efficiency and reliability.

Still curious? Dive deeper

Security & DevOps

The Missing Link in Simplifying Database Complexity

January 19, 2024

6

minutes read

People & Org Design

Building Jelled Teams in Software Development

July 26, 2023

8

minutes read

Coding

Is It Time for You to Switch to Next.js?

March 21, 2024

12

minutes read

Let's get down to business.

Are you searching for a reliable partner to develop your tailor-made software solution? We'd love to chat with you and learn more about your project.