This week’s issue brings you:

• CI/CD Demystified — Build, Test, Deploy, Repeat

• How Do We Design Effective and Safe APIs? (Recap)

• What is MQTT, and How Does It Work? (Recap)

• Database Caching Strategies Explained (Recap)

READ TIME: 5 MINUTES

Pulling Back the Curtain on CI/CD

Some big tech companies like Amazon and Netflix are known to merge code hundreds of times per day.

And it's not just big tech deploying daily. It's hard to find a tech company nowadays, big or small, that doesn't deploy changes frequently.

But this hasn't always been the case.

The infrastructure and tooling to deploy changes frequently only started to come together in the mid-2000s. Before that tools like Git, GitHub, Jenkins, cloud computing, and many more didn't even exist.

In the past, the waterfall model dominated, where each stage of the software development lifecycle (SDLC) unfolded sequentially, resulting in incredibly long release cycles and "integration hell".

Today, software development has transformed into an agile, iterative, and collaborative endeavor. Deployments are frequent, and the process is often streamlined and automated. At the heart of this modern approach to software development lies CI/CD.

What Exactly is CI/CD

CI/CD, or Continuous Integration and Continuous Delivery/Deployment, is a modern approach in software development that emphasizes frequent code integrations and automated testing to ensure code quality.

This approach allows for rapid, reliable, and automated software releases, streamlining the transition from development to deployment in production environments.

By promoting smaller, more frequent updates, CI/CD reduces risks and accelerates feedback loops.

Breaking Down The Twin Pillars of CI/CD

Continuous Integration

CI emphasizes the integration of code changes from multiple contributors into a shared repository (preferably) multiple times a day.

It automates the build, test, and merge process, running unit, integration, and end-to-end tests whenever code is committed.

This frequent amalgamation of changes promotes early detection of any integration issues, allowing teams to address potential problems swiftly and maintain consistent code quality throughout the development process.

Continuous Delivery

Continuous delivery extends the principles of CI, focusing on the automated delivery of integrated code to staging or production environments after passing through the CI process.

This ensures that software is always in a releasable state throughout its lifecycle. Instead of releasing large updates infrequently, CD promotes incremental releases, allowing faster user feedback and more frequent feature rollouts.

Continuous ‘delivery’ and ‘deployment’ are sometimes used synonymously, but there is a clear and important distinction between the two.

Delivery is about ensuring the software can be released at any time. It requires manual intervention to deploy to production.

Deployment, on the other hand, does the release through automated workflows.

The Upsides and Challenges of CI/CD

In today’s fast-paced technology landscape, CI/CD practices can provide significant strategic advantages for businesses and software development teams, including the following:

Speed and efficiency: Automation ensures that the processes run without manual intervention, drastically reducing the time from writing code to deploying it.

Enhanced code quality: Regular integration and testing mean that bugs are identified and rectified early in the process.

Scalability: As your project grows, CI/CD processes scale with it, ensuring consistent performance.

Improved collaboration: With frequent commits and transparent processes, team members can collaboratively review, improve, and finalize changes.

Elevated customer satisfaction: Faster deployments mean users get new features, fixes, and updates more frequently.

While CI/CD offers a plethora of advantages, it's not without its challenges.

Implementing CI/CD requires an initial investment in terms of setting up the required infrastructure and tools. Additionally, it demands a cultural shift, with teams needing to embrace a more collaborative and transparent working style.

For smaller projects or teams, CI/CD might feel like an overkill, introducing complexity where it isn't needed. However, for larger teams and projects that aim to scale, the long-term benefits far outweigh the initial hurdles.

CI/CD in Practice

To implement CI/CD, organizations rely on a combination of tools and cultural practices. There are numerous tools available that cater to different stages of the CI/CD pipeline. But it’s not just about tools, there must also be a culture where developers, testers, and operations teams collaborate closely for CI/CD to work effectively.

Here is an overview of the flow of a CI/CD pipeline:

1) It starts with developers pushing code updates to the version control repository.

2) After that the CI server identifies these updates and initiates the build process.

3) Code is then compiled and is subject to various tests (unit tests, integration tests, and e2e tests).

4) Once the tests have been completed, the outcomes of the tests are relayed back to developers.

5) If tests are successful, the build artifacts are then deployed to staging environments.

6) At this point additional validations, including user acceptance tests or performance tests, can be executed in these staging areas.

7) Once validated and approved, the CD mechanism ensures seamless deployment of changes to the live production environment.

The undeniable benefits of CI/CD, from faster release cycles to higher quality outputs, have solidified its place as a cornerstone in modern software development. The journey of adopting CI/CD may have its challenges, but the rewards are well worth the effort.

API Design Best Practices (Recap)

There are several aspects, techniques, and best practices in API design.

Idempotency, security, versioning, clear resource naming, use of plurals, cross-referencing resources, sorting, and filtering are all aspects that can be observed in the URL.

However, best practices go far beyond what can be observed in API URLs.

Thorough documentation, robust monitoring and logging, consistent error handling, and rate limiting some of the other primary best practices that should be implemented to design effective and safe APIs.

For an explanation of the best practices mentioned above, check out the full post here.

What is MQTT, and How Does It Work? (Recap)

MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol optimized for high-latency or unreliable networks.

Its minimal bandwidth and resource consumption, along with support for real-time data transmission, among other qualities, have made it ideal for IoT environments.

From simple sensors to complex industrial machines, MQTT facilitates efficient data exchange, even in unreliable network conditions.

MQTT operates on top of TCP/IP.

It's focused on simple, efficient, and reliable message exchange.

See the full post here for the core concepts and a breakdown of the MQTT communication workflow.

Database Caching Strategies Explained (Recap)

Five of the most common methods:

Cache-aside strategy — applications control caching by checking the cache before the database. Ideal for dynamic data but requires manual management, increasing complexity.

Write-through strategy — writes simultaneously to the cache and database, ensuring consistency. Suitable for high reliability but increases write latency.

Write-behind strategy — caches data first and updates the database asynchronously. Boosts responsiveness but risks data loss if the cache expires before syncing.

Read-through strategy — automatically caches data on a miss, simplifying code. Initial cache misses can delay retrieval.

Write-around strategy — writes directly to the database, refreshing the cache on read requests. Prevents cache saturation, best for write-intensive applications without immediate read access needs.

For more detailed explanations of each strategy, see the full post here.

That wraps up this week’s issue of Level Up Coding’s newsletter!

Join us again next fortnight where we’ll explore and visually distill more important engineering concepts.