Greetings, fellow engineers! We're back with another issue of Level Up Coding’s newsletter.

In today’s issue:

• Pulling Back the Curtain on CI/CD

• The Secret to Optimizing Your SQL Queries — Execution Order (Recap)

• How OAuth 2.0 Works (Recap)

• How gRPC Works, and Why It’s so Popular for Microservices (Recap)

Read time: 8 minutes

Pulling Back the Curtain on CI/CD

Some big tech companies like Amazon and Netflix are known to merge code hundreds of times per day. And it's not just big tech deploying daily. It's hard to find a tech company nowadays, big or small, that doesn't deploy changes frequently.

But this hasn't always been the case.

The infrastructure and tooling to deploy changes frequently only started to come together in the mid-2000s. Before that tools like Git, GitHub, Jenkins, cloud computing, and many more didn't even exist.

In the past, the waterfall model dominated, where each stage of the software development lifecycle (SDLC) unfolded sequentially, resulting in incredibly long release cycles and "integration hell". Today, software development has transformed into an agile, iterative, and collaborative endeavor. Deployments are frequent, and the process is often streamlined and automated. At the heart of this modern approach to software development lies CI/CD.

What Exactly is CI/CD

CI/CD, or Continuous Integration and Continuous Delivery/Deployment, is a modern approach in software development that emphasizes frequent code integrations and automated testing to ensure code quality.

This approach allows for rapid, reliable, and automated software releases, streamlining the transition from development to deployment in production environments.

By promoting smaller, more frequent updates, CI/CD reduces risks and accelerates feedback loops.

Breaking Down The Twin Pillars of CI/CD

Continuous Integration

CI emphasizes the integration of code changes from multiple contributors into a shared repository (preferably) multiple times a day. It automates the build, test, and merge process, running unit, integration, and end-to-end tests whenever code is committed.

This frequent amalgamation of changes promotes early detection of any integration issues, allowing teams to address potential problems swiftly and maintain consistent code quality throughout the development process.

Continuous Delivery

Continuous delivery extends the principles of CI, focusing on the automated delivery of integrated code to staging or production environments after passing through the CI process. This ensures that software is always in a releasable state throughout its lifecycle. Instead of releasing large updates infrequently, CD promotes incremental releases, allowing faster user feedback and more frequent feature rollouts.

Continuous ‘delivery’ and ‘deployment’ are sometimes used synonymously, but there is a clear and important distinction between the two.

Delivery is about ensuring the software can be released at any time. It requires manual intervention to deploy to production.

Deployment, on the other hand, does the release through automated workflows.

The Upsides and Challenges of CI/CD

In today’s fast-paced technology landscape, CI/CD practices can provide significant strategic advantages for businesses and software development teams, including the following:

Speed and efficiency: Automation ensures that the processes run without manual intervention, drastically reducing the time from writing code to deploying it.

Enhanced code quality: Regular integration and testing mean that bugs are identified and rectified early in the process.

Scalability: As your project grows, CI/CD processes scale with it, ensuring consistent performance.

Improved collaboration: With frequent commits and transparent processes, team members can collaboratively review, improve, and finalize changes.

Elevated customer satisfaction: Faster deployments mean users get new features, fixes, and updates more frequently.

While CI/CD offers a plethora of advantages, it's not without its challenges.

Implementing CI/CD requires an initial investment in terms of setting up the required infrastructure and tools. Additionally, it demands a cultural shift, with teams needing to embrace a more collaborative and transparent working style.

For smaller projects or teams, CI/CD might feel like an overkill, introducing complexity where it isn't needed. However, for larger teams and projects that aim to scale, the long-term benefits far outweigh the initial hurdles.

CI/CD in Practice

To implement CI/CD, organizations rely on a combination of tools and cultural practices. There are numerous tools available that cater to different stages of the CI/CD pipeline. But it’s not just about tools, there must also be a culture where developers, testers, and operations teams collaborate closely for CI/CD to work effectively.

Below is an overview of the flow of a CI/CD pipeline:

1) It starts with developers pushing code updates to the version control repository.

2) After that the CI server identifies these updates and initiates the build process.

3) Code is then compiled and is subject to various tests (unit tests, integration tests, and e2e tests).

4) Once the tests have been completed, the outcomes of the tests are relayed back to developers.

5) If tests are successful, the build artifacts are then deployed to staging environments.

6) At this point additional validations, including user acceptance tests or performance tests, can be executed in these staging areas.

7) Once validated and approved, the CD mechanism ensures seamless deployment of changes to the live production environment.

The undeniable benefits of CI/CD, from faster release cycles to higher quality outputs, have solidified its place as a cornerstone in modern software development. The journey of adopting CI/CD may have its challenges, but the rewards are well worth the effort. As we look to the future of software development, the principles of CI/CD will likely become even more deeply ingrained in best practices.

The Secret to Optimizing Your SQL Queries — Execution Order (Recap)

A SQL query executes its statements in the following order:

1) FROM / JOIN
2) WHERE
3) GROUP BY
4) HAVING
5) SELECT
6) DISTINCT
7)ORDER BY
8) LIMIT / OFFSET

The techniques you implement at each step help speed up the following steps. This is why it's important to know their execution order. To maximize efficiency, focus on optimizing steps earlier in the query.

With that in mind, let's take a look at some optimization tips:

1) Maximize the WHERE clause
This clause is executed early, so it's a good opportunity to reduce the size of your data set before the rest of the query is processed.

2) Filter your rows before a JOIN
Although the FROM/JOIN occurs first, you can still limit the rows. To limit the number of rows you are joining, use a subquery in the FROM statement instead of a table.

3) User WHERE over HAVING
The HAVING clause is executed after WHERE & GROUP BY. This means you're better off moving any appropriate conditions to the WHERE clause when you can.

4) Don’t confuse LIMIT, OFFSET, and DISTINCT for optimization techniques
It's easy to assume that these would boost performance by minimizing the data set, but this isn’t the case. Because they occur at the end of the query, they make little to no impact on its performance.

See here for the original post

How OAuth 2.0 Works (Recap)

OAuth 2.0 is an authorization framework that enables applications to access a user’s data on another service without sharing the user’s password.

It’s essentially a digital handshake between the app, service, and user, with everyone agreeing on what is shared.

The process generally follows 6 steps:

🔶 1) Request access
🔶 2) Redirect to service
🔶 3) Permission request
🔶 4) Authorization code
🔶 5) Exchange code for token
🔶 6) Use the token

There are typically 4 components involved in the process:

🔷 Client (app wanting access)
🔷 Resource owner (user)
🔷 Authorization server
🔷 Resource server

OAuth 2.0 provides multiple grant types to cater to different use cases. These grant types dictate how the application gets an access token. For most web applications, the Authorization Code Grant is the recommended and most secure method to obtain access tokens.

To better understand the process, see here for an extended explanation that walks through an example

How gRPC Works, and Why It’s so Popular for Microservices (Recap)

gRPC is a powerful remote procedure call (RPC) framework developed by Google, enabling efficient and fast communication between services. It is built on HTTP/2 and Protocol Buffers. This is where a lot of the benefits of gRPC are derived from;

🔶 Harnessing Protocol Buffers (Protbufs) as its interface definition language (IDL) helps alleviate tech stack lock-in. Each service can be written in any popular programming language, the IDL works across them all.

🔶 The compact binary format of Protobufs provides faster serialization/deserialization, and a smaller payload than JSON. Less data sent means better performance.

🔶 Since Protobufs are strongly typed, they provide type safety, which can eliminate many potential bugs.

🔶 Utilizing HTTP/2 suppers bidirectional streaming and reduced latency for real-time data transmission.

🔶 The combination of HTTP/2 and Protobufs provides maximum throughput and minimal latency. It's often faster than the traditional JSON over HTTP approach.

The easy implementation and benefits above have made gRPC very popular for microservices communication.

Check the full post here which also covers why gRPC isn’t used between web clients and web servers

That wraps up this week’s issue of Level Up Coding’s newsletter!

Join us again next week where we’ll explore optimizing queries with database indexing, data structures in real life, and load balancer vs reverse proxy.