Hírek, események

Elixir in Production: What Is It Used For?

We’ve already written a series of articles about why we think Elixir is a great choice, but don’t take our word for it – there are many success stories out there about using Elixir in production that not only prove that the language is mature enough to be a solid choice, but it can be even more effective than the usual frequently used languages and frameworks thanks to the features provided by BEAM and OTP.

From startups to established enterprises, our examples clearly outline Elixir’s strengths:

  • Scalability – Effortlessly handles sudden surges in traffic and data.
  • Fault Tolerance – Maintains stability and uptime even during system failures.
  • Cost Efficiency – Reduces infrastructure needs.

After understanding how companies like Discord integrated Elixir to handle real-time communication and how Pinterest gained significant cost savings with it, we hope you’ll find Elixir inspiring enough to take a look at.

Incredible Developer Productivity with Elixir at Remote.com

Remote.com is built on Elixir from the ground up, for good reasons. 

“It offers incredible developer productivity due to its intuitive and straight-forward syntax, its well-designed standard libraries, and its convenient level of abstraction.“ – according to Peter Ullrich, a Senior Elixir Engineer at Remote.

“We rarely have to reinvent the wheel,” says Peter, because Elixir provides them with the tools they need to improve their services while maintaining their desired productivity levels at the same time.

It’s no wonder why Elixir became a serious player in the webdev space:

  • Phoenix, its web framework is one of the best choices for writing webapps.
  • It’s concurrency-focused design allows effortless scalability.
  • The Erlang VM (BEAM) is built in a “let it crash” philosophy, thatencourages developers to build applications that can gracefully handle failures and self-recover.

When it comes to the ecosystem and particular use cases, Elixir proved to be a great choice for Remote.com.  For data processing they use Broadway, for multimedia processing Membrane. 

“Its recent ventures into machine learning (Nx) and collaborative development (Livebook) put Elixir on the map of many more industries and allowed it to be used for a whole range of new use-cases.”

“It Just Works” – Elixir by Accident at Multiverse 

Most companies on this list chose Elixir after careful consideration and because of its clear advantages. Multiverse, a UK-based Ed-Tech startup (with a $220M Series D behind them), is built with Elixir simply because when the company was founded, an external agency was contracted to build the initial platform. And they happened to be an Elixir shop.

However, while most companies seem to try to get rid of their initial platform by rebuilding it from the ground up, or just barely keeping it running with endless “quick fixes”, Multiverse considers itself very lucky to have Elixir as the foundation by this “lucky accident.”

Razvan, a Senior Engineering Manager at Multiverse, is confident that it will help Multiverse scale, and they – too – are in contact with the creators of the language and the frameworks they use. Although the community is small compared to other languages, it is very active and supportive. 

This is something we especially love about Elixir at RisingStack, too.

“I’m coming from a JavaScript background and I can say that the first thing that amazed me is that Elixir, well, just works.”

When Razvan joined the team with little to no experience, he was able to get their Elixir platform up and running in a couple of hours without any glitches.

  • The environment feels robust, cohesive and you don’t waste hours and hours trying to run it, to debug a cryptic error.
  • The Elixir docs are amazing. It is actually useful, to the point and with examples. 
  • The tooling is powerful and testing is a first class citizen of Elixir development. 
  • Functional programming is just awesome!

On top of that, the frameworks aren’t too bad either! 

“We’re currently using Elixir with Phoenix and LiveView, and our engineers are very happy with that, mostly because they don’t have to switch between languages all the time.” – said Razvan.

Less Servers, Same Performance at Pinterest

Pinterest estimates that it has saved about $2 million per year in server costs since it successfully adopted Elixir.

Security, scalability, and fault tolerance are all important aspects of a system, but we can’t ignore the economical side of keeping it running – we have to think about costs too. The engineering team at Pinterest tried to find a solution for all of the above, and Elixir provided it for them. 

Pinterest managed to replace 200 servers running Python with just 4 running on Elixir, while providing the same performance as before. Besides, maintenance became much easier as well.

Here are some highlights from the interview for those who are in a rush:

  • Pinterest chose Elixir because they were looking for a system that was easy for programmers to understand.
  • Elixir’s main strengths: friendly syntax, powerful metaprogramming features, and incorporation of the Actor model.
  • Besides the cost saving, the performance and reliability of the systems went up despite running on drastically less hardware.
  • In total, Pinterest reduced its server size by about 95% thanks to Elixir.
  • Despite running on less hardware, the response times dropped significantly, as did errors.

How Two Elixir Nodes Outperformed 20 Ruby Nodes by 83x

Veeps is a streaming service that also offers ticket-based online events where it’s not uncommon for fans to immediately jump on ticket sales once an event is announced.

While the previous Ruby on Rails system was able to service thousands of users, or even tens of thousands, the engineering team found it next to impossible to scale it further to allow more visits while maintaining performance at the same time.

Vincent Franco, the CTO, convinced management to make the switch to Elixir to future-proof the company’s tech stack. It took about 8 months to rewrite everything in Elixir and Phoenix, but it turned out that the effort was worth it.

Two Elixir nodes replaced 20 Ruby on Rails nodes, and those two can handle 83x more users than before. The project was managed by an external company due to a lack of internal experience with Elixir, but by the time it was finished, Veeps was able to establish an internal team that can build and expand on the newly built Elixir-based systems.

Elixir Powers Emerging Markets – Literally

Access to electricity is a given in developed countries, but still a rare commodity in other parts of the world. SparkMeter aims to change that with their grid-management solutions. They operate smart meters that communicate with a grid management unit which is connected to cloud servers.

This may not sound like an unsolvable challenge so far, but there are additional complexities. The servers and the grid management unit communicate via cellular network which is prone to failure, and the electricity powering the systems may also go down time-to-time. Fault-tolerance was crucial in circumstances like this, And Elixir together with the Nerves platform is the perfect choice for this situation. 

Nerves is an open-source platform that combines the BEAM virtual machine and Elixir ecosystem to easily build and deploy embedded systems for production. A highlight of this setup is that it can handle cases when parts of the system are down.

Finding capable engineers was also a problem, but after the project started with outside consultants, their in-house team was trained in Elixir to be able to maintain and further develop the system as a long-term solution which worked out really well.

In the new system, the grid management unit communicates with the meters via radio, using Rust for hardware control and Elixir Ports for data processing. Communication with cloud servers over 3G or Edge required a custom protocol to minimize bandwidth usage, crafted uniquely to fit their specific needs.

Additionally, their system includes a local web interface accessible via Wi-Fi, using Phoenix LiveView, and a cloud-based system that processes data through a custom TCP server and a Broadway pipeline, storing it in PostgreSQL. This robust setup allows SparkMeter to maintain high availability and manage its resources efficiently, despite the challenging environments it operates in. The team also managed to reduce the complexity of the previous architecture by replacing the old one that was using Ubuntu and Docker for the system level, Python/Celery and RabbitMQ for asynchronous processing, and Systemd for managing starting job processes with just Elixir and Nerves.

Multiplayer With Elixir: 10000 Players in the Same Session

X-Plane 11 is one of the best flight simulators in the world, aiming for unparalleled accuracy for even pilots to practice in a safe environment. It used to have a simple peer-to-peer solution for multiplayer, but the developers wanted to change that.

This proved to be a huge challenge: the team did not have experience in the subject, but they needed a solution that could support way more concurrent players than an average multiplayer game, and it needed to do that with as much accuracy as possible.

With a criteria like this, simply adding more servers was not an adequate solution so they excluded Ruby and Python among others. The top three choices were Rust, Go and Elixir: Elixir won because of its fault-tolerance capabilities and predictable latency. In addition to that, Elixir and Erlang have built-in support for parsing binary packets which made the implementation easier.

The whole project took only 6 months, and that included the time to learn Elixir because the lead developer had no prior experience with the language.

It’s so lightweight that the entire player base in North America is served by a single server running on 1 eight-core machine with 16GB of memory. 

The solution is open-source if you want to take a look.

How PepsiCo Uses Elixir

Many companies internally build their own tools for specific purposes, and PepsiCo is no exception. The Search Marketing and Sales Intelligence Platform teams handle immense amounts of data coming from their search partners that needed a support of a robust platform.



This complex data pipeline is managed by the Data Engineering team, which initially collects and stores the data in the Snowflake Data Cloud. An Elixir application then processes this data and routes it to PostgreSQL or Apache Druid, depending on its characteristics. A Phoenix application serves this processed data to internal teams and interacts with third-party APIs.

The team used Elixir to create a domain-specific language that translates business queries into data structures. It’s easy for them to extend it as they need and provides a solid foundation, even when connecting to several different third-party APIs.

David Antaramian, a Software Engineering Manager at PepsiCo praises the Erlang runtime and its standard library, particularly for managing large datasets without frequently accessing the database. He emphasizes the use of Erlang’s in-memory table storage, ETS, which allows them to efficiently store hundreds of thousands of rows. This capability is crucial for handling the massive amounts of data PepsiCo deals with.

The Elixir ecosystem effectively complements Erlang, supporting both front-end and server-side operations at PepsiCo. The company’s front-end, built in React, connects with the server using the Absinthe GraphQL toolkit atop the Phoenix web framework. For database interactions, the Ecto library manages communications with PostgreSQL. Additionally, the esaml and Samly libraries are used for authentication across PepsiCo’s network, demonstrating a practical application of tools from the Erlang and Elixir communities.

4 Billion Messages Every Day on Discord With Elixir

Discord is one of the most famous adopters of Elixir since they have been building the platform on it from day one. Beside a Python API in a monolith architecture, Discord has about 20 different services built with Elixir.

However, this choice was not without risk. In 2015, when Elixir v1.0 came out and Discord was founded, the team gambled and bet on Elixir, hoping the language would mature and evolve over time. Their bet paid off wonderfully, as seen from the growing user base.

Jake Heinz, Lead Software Engineer at Discord said about Elixir: “In terms of real-time communication, the Erlang VM is the best tool for the job. It is a very versatile runtime with excellent tooling and reasoning for building distributed systems”. 

Those 20+ services are powered by 400-500 Elixir servers, and amazingly only maintained by a handful of engineers.

The team uses Distributed Erlang, facilitated by etcd for service discovery and configuration, to create a partially meshed network rather than a fully connected one. This setup allows for efficient and scalable communication across Discord’s numerous services, including their audio and video platforms, which operate over 1000+ nodes. When needed, engineers from other teams can collaborate with the Chat Infrastructure Team operating these services and build on it with their assistance.

Discord also uses Rust to complement Elixir, with the help of the Rustler project to bridge the gap between the two languages by hooking a custom data structure built in Rust directly into the Elixir servers. The flexibility of it allows engineers to solve uptime-related problems often in just a few minutes

Although none of the engineers had prior experience with Elixir before joining Discord, they quickly pick up the pace and thanks to Erlang VM, even able to efficiently debug a live system if needed.

Conclusion

As seen from these case studies, most companies needed not only scalability but also ease of maintenance and future-proofing – Elixir was able to provide all of these, thus proving its maturity for an environment that is more than ready for use in production.

Our team also works with Elixir more and more often, utilizing it in situations where mighty JavaScript falls short.

If you need advice with Elixir, reach out to us for a chat!

Practical Tutorial to Retrieval Augmented Generation on Google Colab

We will take a look at how to set up a RAG – Retrieval Augmented Generation – demo with the Anthropic Claude 3 Sonet model, using google’s CoLab platform. CoLab offers free instances with T4 GPUs sometimes, but we’ll only need a simple CPU instance, since we access the model only through API.

RAG can be used to update already trained models with new information to improve its question answering capabilities. We will be loading the new data into a vector database that will serve as an additional, external memory for the model. This will be accessed by a retrieval model – llama-index in our case – that constructs a task specific prompt and fetches the document, passing both on to the language model.

Dependencies

Grab something to enhance the model with – we’ll use a paper about QLoRA – Quantized Low Rank Adaptation – for this example, but this could be any text based content that was not part of the training for the particular model. Since this is a pdf, we’ll need to use a pdf loader later, make sure to account for that in case you want to use some other format.

We can use shell commands by prefixing them with an exclamation mark in the notebook. Using this makes it simple to download the source pdf:

!wget "https://arxiv.org/pdf/2305.14314.pdf" -O /content/QLORA.pdf

Note: for some reason I don’t fully understand yet, I had to open the pdf in my browser before CoLab could download it, otherwise I got 403 errors until I did.

There are some python dependencies we will also need to install:

!pip install torch llama-index==0.10.20 transformers accelerate bitsandbytes pypdf chromadb==0.4.24 sentence-transformers pydantic==1.10.11 llama-index-embeddings-huggingface llama-index-llms-huggingface llama-index-readers-file llama-index-vector-stores-chroma llama-index-llms-anthropic --quiet

The model

There are quite a few things we will need to import from the packages we just installed. Not only that, but we will have to register an account with Anthropic to be able to access their models, since they are not open source. They do, however, offer $5 worth of API usage for free, of which we’ll need about 3 cents for this demo. Feel free to spend the rest on whatever else you’d like to test with it! You can register an account with Anthropic here.

## app.py

## Import necessary libraries
import torch
import sys
import chromadb
from llama_index.core import VectorStoreIndex, download_loader, ServiceContext, Settings
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.core.storage.storage_context import StorageContext
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.readers.file import PDFReader
from llama_index.llms.anthropic import Anthropic
from transformers import BitsAndBytesConfig
from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList
from llama_index.core import PromptTemplate
from llama_index.llms.huggingface import HuggingFaceLLM
from IPython.display import Markdown, display, HTML
from pathlib import Path
import os

After registering and activating the $5 voucher, we will need to create an API key and then add it as an environment variable to the code as well.

os.environ["ANTHROPIC_API_KEY"] = "YOUR API KEY HERE"

Then, load the pdf we downloaded that contains details about QLoRA:

loader = PDFReader()
documents = loader.load_data(file=Path("/content/QLORA.pdf"))

The setup for the model itself is pretty simple in this case, since Anthropic models are not open source, and we can only interact with them through their API:

llm = Anthropic(
    model="claude-3-sonnet-20240229",
)

tokenizer = Anthropic().tokenizer
Settings.tokenizer = tokenizer
Settings.llm = llm
Settings.chunk_size = 1024

First, we ask the model about QLoRA to see if it possesses any knowledge on this topic:

# resp contains the response
resp = llm.complete("What is QLORA?")

# Using HTML with inline CSS for styling (gray color, smaller font size)
html_text = f'<p style="color: #1f77b4; font-size: 14px;"><b>{resp}</b></p>'
display(HTML(html_text))

QLORA is not a commonly recognized acronym or term that I'm familiar with. Without more context, it's difficult for me to provide a definitive explanation of what QLORA means or refers to. Acronyms can have multiple meanings across different fields or contexts. Could you provide some additional details about where you encountered this term or what domain it relates to? That would help me try to determine the intended meaning of QLORA.

New context and questions

As you can see from the following output, the model doesn’t have data on this topic, which is great for us, as we can now attempt to extend its knowledge on the subject using RAG. We’ll now set up ChromaDB as our vector database, and load the data from the downloaded paper to it. Chroma is an open-source vector embedding database. When queried, it will compute the feature vector of our prompt and retrieve the most relevant documents – the one we will load into it – using similarity search, so the document can be then passed to the language model as context. You don’t need to attach any external servers, as ChromaDB can run within our Jupyter Notebook and was installed in the beginning with pip.

#Create client and a new collection
chroma_client = chromadb.EphemeralClient()
chroma_collection = chroma_client.create_collection("firstcollection")

# Load the embedding model
embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-base-en-v1.5")

# Set up ChromaVectorStore and load in data
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
service_context = ServiceContext.from_defaults(llm=llm, embed_model=embed_model)
index = VectorStoreIndex.from_documents(
  documents, storage_context=storage_context, service_context=service_context
)

Chroma will not have a hard time figuring out which document to return since we only use one, but you can definitely play around with loading additional documents and seeing how that affects the result. For now though, we will just ask the same question again:

#Define query
query="what is QLORA?"

query_engine =index.as_query_engine(response_mode="compact")
response = query_engine.query(query)

# Using HTML with inline CSS for styling (blue color)
html_text = f'<p style="color: #1f77b4; font-size: 14px;"><b>{response}</b></p>'
display(HTML(html_text))

But now we get a different answer:

QLORA stands for Quantized Low-Rank Adaptation. It is a technique for efficiently finetuning large language models by only updating a small set of parameters during training, rather than the full model weights. This allows for significant memory savings compared to standard full model finetuning. QLORA uses quantization to further reduce the memory footprint, with model weights stored in a low-precision 4-bit format during training. The key components are low-rank adaptation using LoRA layers, and quantization using a custom 4-bit numeric format called NormalFloat4. This enables finetuning of very large models like GPT-3 on a single GPU, which would not be feasible with full precision finetuning.

This seems like a pretty good quality answer, and in-line with the paper we used as context. This example demonstrates how we can use RAG to feed new information to a model to keep it up to date without having to re-train it on an even larger dataset from the beginning.

Now that we confirmed that the new context is indeed being used, we can also ask something else that should be included in the paper:

chat_engine = index.as_chat_engine(chat_mode="condense_question", verbose=True)
response = chat_engine.chat("What would be potential real world use cases for QLoRA?")
print(response)

And the response:

Querying with: Given the previous conversation, what would be potential real-world use cases or applications for QLoRA (Quantization-aware Learned Optimized Residual Addition), which is a technique for quantizing and compressing large language models like GPT while maintaining their performance?
Based on the information provided, some potential real-world applications and use cases for QLoRA could include:

1. Enabling efficient finetuning and deployment of large language models on resource-constrained devices like mobile phones, IoT devices, or edge computing systems with limited memory and compute capabilities.

2. Facilitating the use of state-of-the-art large language models in cloud services or web applications where memory and computational efficiency is crucial for scalability and cost-effectiveness.

3. Allowing researchers and developers to experiment with and finetune very large language models (e.g., 65B parameters) on modest hardware like a single GPU, accelerating research and development in natural language processing.

4. Reducing the carbon footprint and energy consumption associated with training and running large language models, making them more environmentally sustainable.

5. Enabling the deployment of high-performance language models in embedded systems, robotics, or other specialized hardware with strict memory and compute constraints.

The key advantage of QLoRA seems to be its ability to compress and quantize large language models to a 4-bit representation while preserving their performance through efficient finetuning techniques like Low Rank Adapters (LoRA). This could unlock a wide range of applications where state-of-the-art language models were previously impractical due to resource constraints.

While the model couldn’t quite figure out what QLoRA stands for – the source paper did not have the explanation for it, so it’s hardly a surprise – the response ended up being a pretty solid list of pros.

Summary

From this small scale RAG demo, we can see how easy it can be to enhance the memory of language models in a performant and relatively resource efficient manner – the vector database will still require space to store all the new information, but no GPU is needed for just the process itself. In the example, we used an externally hosted model, but you can use other pre-trained models that can run in CoLab, provided you can get a hold of one of those elusive GPU instances.

Elixir Cheatsheet: A Node.js Developer’s Transition Guide

We’ve previously written about the reasons for trying Elixir out, as well as a how-to-get-started guide. However, there is still a long way ahead of you after firing up your thrusters. While the tutorial and documentation of both Elixir and Phoenix are the best I’ve ever seen by a great margin, the world of BEAM and OTP will be quite alien to what you are used to as a JS/TS developer. To make the embracing of unfamiliar concepts easier, we’ve created a cheatsheet to serve as an anchor during your journey.

The basics: Variables, Data Types, and Immutability and More

We start with the basics: primitive types, lists, and maps. The basic building blocks of applications in any language. This section acts as a quick reference for those familiar with JavaScript but new to Elixir’s strange and exciting ways.

One of the first things you’ll notice is the difference in variable handling. Unlike JavaScript, variables in Elixir are immutable, meaning their values cannot be changed after assignment. This enforces a safer programming style and eliminates the risk of unintended side effects.

Elixir also uses atoms instead of symbols, and thanks to pattern matching, they often also replace booleans. These atoms are lightweight, immutable entities that are perfect for representing simple data like true/false or unique identifiers.

String manipulation should feel familiar, with interpolation supported using double quotes and character escapes similar to JavaScript. However, Elixir offers sigils for string creation, providing flexibility depending on your needs.

For data structures, Elixir provides lists and tuples, similar to arrays in JavaScript. Lists are implemented as linked lists, so they are great for storing variable length data when you might need to append (and not push!) new elements to the collection, while tuples have defined size and thus are used for fixed data. 

Maps, on the other hand, resemble JavaScript objects but allow any data type, not just strings, as keys. This flexibility makes them powerful tools for storing and organizing diverse data.

Beyond these fundamentals, Elixir offers keyword lists and the enumerable protocol. Keyword lists provide a concise way to associate data with keywords, while the Enum module offers powerful functions for common operations on enumerable data structures.

Control Flow

Elixir offers a familiar-looking if-else and case and with blocks for controlling the flow of your app. But following in the footsteps of Ruby, we also have,  unless, if we’d prefer not writing not or !. We also have an old friend you might recognize. Don’t be fooled though, as they are not what they seem! When comparing these keywords with JavaScript, mostly everything is a false friend due to Elixir’s expression-oriented nature, so you’ll need to come back to this section often in your first weeks. 

Functions

Elixir’s functions come in various flavors and attached goodies, offering flexibility and expressiveness in your code. In the cheatsheet, you can find examples for:

Anonymous Functions:

  • Defined with fn and end keywords.
  • Used for short, one-time operations.
  • Can be assigned to variables for later use.

Module Functions:

  • Defined within modules using def or defp (private) keyword.
  • Public by default, requiring explicit marking as private if needed.
  • Offer clear organization and separation of concerns.

Function Signatures:

  • Described using Module.function_name/arity, where arity is the number of arguments.
  • Can be overloaded based on argument number or type using pattern matching.

Overloading Functions:

  • Achieved through pattern matching, guards, or default arguments.
  • Allows defining multiple functions with the same name but different argument combinations.

Pipe Operator (|>)

  • Chains function calls together, automatically passing the previous output as the first argument to the next function.
  • Offers concise and readable function chaining.

Capture Operator (&)

  • Captures functions into anonymous functions.
  • Useful for creating anonymous functions from existing named functions or passing functions as arguments.

Remember, these are just some of the fundamentals of functions in Elixir. The language offers further features like recursion and higher-order functions for building complex and elegant solutions, which would be too much to capture in a cheatsheet.

Pattern matching

Pattern matching is probably the most powerful and convenient property of functional languages. A cheatsheet cannot do justice to it, but at least you can come back and see the related language constructs:

  • Match operator (=): Its usage for variable assignment, value comparison, and data deconstruction (maps, lists, structs, etc.).
  • Function invocation: How pattern matching is used to identify the correct function based on factors like module name, function name, arity, argument types, default arguments, and function guards.
  • case expressions: Utilizing pattern matching for conditional branching.
  • Pin operator (^): Preventing variable reassignment during pattern matching within functions.

Modules and Structs

ES Modules and Elixir modules are similar only in name, so we included a lengthy explanation of them in the cheatsheet. We tried to make order regarding alias, require, import and use, because those can be confusing at first. Hint: require has nothing to do with it’s JS counterpart. But you probably guessed it by now. 

You can also find examples for Structs. While Elixir is not a statically typed language like TypeScript, Structs can provide the necessary type definitions when needed. Used in tandem with pattern matching, you’ll soon realize that type safety is overrated, especially when you feel how efficient you can be when you can forgo type wrangling and gymnastics, that is so common when writing TS.

Ready to Dive In?

We stand by what we said earlier: learning Elixir start paying dividends quickly. However, the first steps can be daunting, as there are new concepts you need to get familar with, and the syntax can feel alien at first. But whether you’re exploring Elixir for a specific project or keen on expanding your programming repertoire, we hope this cheatsheet will prove useful when you make the leap from Node.js to Elixir.

Where to Host Your AI: Comparing ML Model Deployment Services

When it comes to hosting machine learning models, whether it is for private or public use, it’s not a simple task to find the right services for the job. Many articles online and responses from AI tools tend to include a wide range of tools, platforms and providers that have only one thing in common, being related to machine learning.

In this post, we aim to help by providing a list of services that actually make hosting ML models possible, curated by hand.

Modal is an ML model hosting and training platform, that has direct code integration for runtime configuration, as well as a CLI tool for initiating deployments.

The main features offered include cron jobs for task scheduling, log collection and retention, monitoring, webhook endpoints, secret management and support for custom built images and custom domains. Modal’s infrastructure also supports distributed queues, distributed dictionary learning, and CPU and GPU concurrency.

Everything related to the runtime environment is configured in python, no separate containerisation technology necessary, although working knowledge of docker can be useful seeing how similar the actual configuration is to the structure of a Dockerfile. Having the environment configuration as part of the code can have downsides as well however, as it will have to be built at runtime resulting in potentially longer and more expensive runs. The image, once built is then stored by Modal, so it can be re-used without the need to rebuild.

Modal offers three pricing tiers, from a free tier to team and enterprise subscriptions. The free and the team tiers include 30$ of compute credit a month. The team subscription is 100$ per month, and comes with 10 seats, with the possibility to pay for additional seats at 10$ per. Enterprise subscription details are individually determined, but all of them appear to have no limitation on seats. You can find the currently listed compute costs in the following table:

Hardware typeCost
CPU$0.192 / core / h
Nvidia A100, 40 GB VRAM$3.73 / h
Nvidia A100, 80 GB VRAM$5.59 / h
Nvidia A10G$1.10 / h
Nvidia L4$1.05 / h
Nvidia T4$0.59 / h
Memory$0.024 / GiB / h

Paperspace

Paperspace offers access to many GPU types and deployments configurable from their web UI requiring minimal setup. While it uses docker images, it can pull any public image by providing a url, and can also be set up to use custom images from private registries. Models can be pulled from S3 buckets or from Huggingface. When it comes to high availability, it is possible to create multiple replicas when setting up a deployment and further autoscaling can also be configured here.

Aside from model deployments, it is also possible to create Jupyter notebooks, set up model training workflows and manage secrets on the web UI, but Paperspace has an open source CLI tool with full access to its features if you’d rather.

As far as subscription goes, Paperspace offers four tiers of subscriptions with more powerful instance types becoming available as prices get higher, as well as private projects for all paid tiers. A free tier suitable for trying out the platform is available, with a project limit of 5, 5GB of free storage space and no concurrent job runs.

Paid tiers start at 8$ for a single seat pro tier and 12$ for a team of 2, including a cap of 10 projects, 15GB of free storage and 3 concurrent jobs. At 39$ per seat, the growth tier has a cap of 5 seats, a project limit of 25, free storage of 50GB and 10 concurrent jobs. An enterprise tier is also available, with costs and limits up to an individual contract. While it is possible to go over the free storage limit, overages are billed at $0.29/GB.

We summarised the current prices of compute instances available in the following table:

Instance typeHardwareCostAvailable in free tier
C4 CPU2 CPU 4GB RAM$0.04 / hyes
C5 CPU4 CPU 8GB RAM$0.08 / hyes
C7 CPU12 CPU 30GB RAM$0.30 / hyes
P4000 GPU8 CPU 30GB RAM 8GB VRAM$0.51 / hyes
RTX4000 GPU8 CPU 30GB RAM 8GB VRAM$0.56 / hyes
A4000 GPU8 CPU 45GB RAM 16GB VRAM$0.76 / hyes
P5000 GPU8 CPU 30GB RAM 16GB VRAM$0.78 / hyes
P6000 GPU8 CPU 30GB RAM 24GB VRAM$1.10 / hyes
A5000 GPU8 CPU 45GB RAM 24GB VRAM$1.38 / hno
A4000 GPU x216 CPU 90GB RAM 16GB VRAM$1.52 / hyes
A6000 GPU8 CPU 45GB RAM 48GB VRAM$1.89 / hyes
v100 GPU8 CPU 30GB RAM 16GB VRAM$2.30 / hno
V100-32G GPU8 CPU 30GB RAM 32GB VRAM$2.30 / hyes
A5000 GPU x216 CPU 90GB RAM 24GB VRAM$2.76 / hyes
A100 GPU12 CPU 90GB RAM 40GB VRAM$3.09 / hno
A100-80G GPU12 CPU 90GB RAM 80GB VRAM$3.18 / hyes
A6000 GPU x216 CPU 90GB RAM 48GB VRAM$3.78 / hyes
V100-32G GPU x216 CPU 60GB RAM 32GB VRAM$4.60 / hno
A100 GPU x224 CPU 180GB RAM 40GB VRAM$6.18 / hno
A6000 GPU x432 CPU 180GB RAM 48GB VRAM$7.56 / hno
V100-32G GPU x432 CPU 120GB RAM 32GB VRAM$9.20 / hno

These prices are in addition to the monthly subscription, with free credit offered on a case-by-case basis.

Self-managed Ray

Ray is an open source framework encompassing many tools ranging from libraries to help with common machine learning tasks to distributed computing and parallelisation, from the deployment of ML models to training them and running workloads on them. Ray supports python when it comes to its developer tools, but the deployment and running of models should be usable pretty much anywhere, even locally on a laptop computer – although the computer used should still have a GPU if the model requires it.

It is possible to host Ray on many cloud platforms, being an open source project, with official Ray cluster integrations available for AWS and GCP, and community maintained integrations for Azure, Aliyun and vSphere. Ray also offers configuration files for running it inside a kubernetes cluster via kuberay, enabling it to be hosted with any cloud provider that supports kubernetes.

The cost of a self-managed Ray cluster mainly comes down to the pricing of the chosen cloud platform and the work required to set up and maintain the cluster and related infrastructure, but this option affords the most flexibility and customisability.

Anyscale – managed Ray

Anyscale offers a managed Ray solution on top of the biggest cloud providers, and is operated by the core team behind the development of Ray itself. Even though Ray supports Kubernetes and using Docker images, Anyscale uses plain vms but still provides logs and grafana for monitoring from their own UI. The UI also allows for the launch of workloads and configuration of additional environments through workspaces. Integrations are available for vscode and jupyter notebook to allow developers to launch workloads right from their development tools.

Unfortunately, Anyscale has not published any pricing information for their managed Ray offering, you’d need to contact sales to get a quote. Additional costs with the chosen cloud platform provider should also be considered – AWS and GCP are supported while Azure at the moment is not.

Amazon SageMaker

SageMaker allows for building, training, and deployment of machine learning models using Amazon’s existing infrastructure and some new ML tools. Among many others, it features an IDE, SageMaker Studio for development and deployment and a model management tool called SageMaker MLOps. SageMaker Serverless Inference is a serverless option for serving models that doesn’t require choosing an instance type.

Amazon claims that “SageMaker offers at least 54% lower total cost of ownership (TCO) over a three-year period compared to other cloud-based self-managed solutions”. To help with figuring out costs, there is table detailing the prices of available instance types included on the official page here. SageMaker Serverless Inference prices are based on the duration of the inference and the amount of data that has been processed.

HuggingFace Inference Endpoints

HuggingFace is the biggest ml model and dataset repository out there, and they also offer their own production ready hosting solution in the form of serverless endpoints. Although it does not seem to affect pricing, you can choose which cloud provider to use for your endpoint, AWS, GCP or Azure, each with multiple possible regions.

Further configuration allows for defining minimum and maximum replicas to control autoscaling, with the possible minimum value of 0 meaning that the endpoint will be able to scale down completely when not in use. This has the potential to save quite an amount of money, at the cost of increased waiting time when calling the scaled down endpoint, as it has to spin up an instance before beginning to process the request. It is also possible to configure ssl for your endpoint, or make it entirely private if you choose to.

Before you deploy your model, HuggingFace gives you an estimated monthly cost based on chosen hardware, assuming that the endpoint will be up for the whole month, but excluding any scaling. This can be quite handy to have an idea about just the baseline cost of having a model deployed. Once finished with the configuration, you get a url where yoy can access the model, as well as an inference widget that allows you to test the endpoint.

Pricing is tied to instance types, and while there are different paid services offered by HuggingFace, Inference Endpoints is the one to look out for when considering model hosting.

Available GPU instances on aws:

GPUMemoryPrice
NVIDIA T414GB$0.60 / h
NVIDIA A10G24GB$1.30 / h
NVIDIA T4 x456GB$4.50 / h
NVIDIA A10080GB$6.50 / h
NVIDIA A10G x496GB$7.00 / h
NVIDIA A100 x2160GB$13.00 / h
NVIDIA A100 x4320GB$26.00 / h
NVIDIA A100 x8640GB$45.00 / h

CPU instances are available both on aws and azure, with the same hourly rates:

vCPUMemoryPrice
1 Intel Xeon core2GB$0.06 / h
2 Intel Xeon cores4GB$0.12 / h
4 Intel Xeon cores8GB$0.24 / h
8 Intel Xeon cores16GB$0.48 / h

Compared

ModalRayAnyscaleSageMakerPaperspaceHuggingFace
Open-sourcenoyesnononono
Can be used locallynoyesnononono
Vendor lock-inModalnoAWS or GCPAWSPaperspace / DigitalOceanAWS, GCP or Azure
Containersfrom Pythonyesyesyes / from UIyes / from UIyes / from UI
Zero-opsfrom Pythonnomostly UImostly UImostly UImostly UI
Pricingtransparentcloud provider dependentnot disclosedtransparent but complicatedtransparenttransparent
Easy to startif you know Dockeryesnonoyesyes
Free tieravailable with 30$ credithosting dependent, can also be run locallypossible, contact salespossible, contact salesavailable, free credit offered in confirmation emailonly hub is free

All in all, Modal could be a pretty good place to start, with a little image building to get comfortable with, but all of that is done in python, and it has clear pricing that is easy to calculate.

Ray is open source and is the most flexible choice, but will likely require dedicated engineers to set up and maintain.

Anyscale could be a great and simple managed solution for Ray, but with no public pricing, it really depends on what kind of deal you get.

As for SageMaker, while it has pricing information, it is as complicated as it gets with AWS to actually figure out how much you’ll end up paying for it. It has a whole ecosystem of tools for everything you might need in one place accessible from a web UI, with the possibility to connect any other AWS service on top – knowing AWS you’ll probably have to use a bunch of their other services eventually.

Paperspace’s 2023 acquisition sounds like a good opportunity for DigitalOcean to expand into the AI platform market, with an easy to start but still fairly customisable offering. The current prices on their instances seem better than the competitors, with the subscription fees generally being higher.

Then, there is also HuggingFace, the de-facto model repository, offering the most commonly used GPUs for model hosting at competitive prices and simple configuration that can be done from their UI as well.

Nuxt 3 Caching with Authentication

We had a project where we aimed to optimize page load times while preserving SEO benefits. One of the techniques we employed was enabling ISR (Incremental Static Regeneration), which caches the page’s HTML response on the CDN network until the TTL (Time to Live) expires. However, we also encountered challenges with parts of the pages that were user-specific, such as profile data and the number of items. These components couldn’t be cached, as doing so might result in one user seeing another user’s items. Addressing this issue is the focus of our article.

Project setup

Pages

The project has 5 pages with different rendering modes enabled:

  1. SSR
  2. ISR without TTL
  3. ISR with TTL
  4. SWR without TTL
  5. SWR with TTL

To learn more about rendering modes in Nuxt 3, check out our blogpost here.

Example page code:

<template>
    <div>
        <p>{{ pageType }} page</p>
        <pre>Time after hydration: {{ new Date().toUTCString() }} </pre>
        <pre>Time in server rendered HTML: {{ data }}</pre>
        <NuxtLink to="/">Home</NuxtLink>
    </div>
</template>
<script setup lang="ts">
const pageType = "SSR"; // value differs based on route
const { data } = await useFetch('/api/hello')
</script>

Rendering modes

Rendering modes are set up in nuxt.config:

export default defineNuxtConfig({
  ssr: true,
  routeRules: {
    "/isr_ttl": { isr: 60 },
    "/isr_no_ttl": { isr: true },
    "/swr_ttl": { swr: 60 },
    "/swr_no_ttl": { swr: true },
  },
});

Database

For the purpose of this showcase we use Vercel KV for a simple db-like functionality.

Note Originally, we intended to use a simple object to mimic database functionality as shown in the following code snippet:

dbFake.ts:

export const users = [
  {
    id: 1,
    loggedIn: false,
  },
];

However, this approach only works in local development and is not suitable for deployment on platforms like Vercel or Netlify, where serverless/edge functions are employed. In such environments, the server does not run continuously. Instead, a lambda function is started and then stopped whenever there is an API request. Consequently, an object on the server side cannot preserve its state.

Server routes

The server has 4 routes:

api/hello

Route simply returns a current date:

export default defineEventHandler((event) => {
  return  new Date().toUTCString();
});

api/auth

This route returns the loggedIn status of the first user:

import { kv } from '@vercel/kv';

export default defineEventHandler(async (event) => {
  const loggedIn = await kv.hget("user1", "loggedIn");
  return { loggedIn };
});

api/login

This route updates the loggedIn status to true and returns this value:

import { kv } from "@vercel/kv";

export default defineEventHandler(async (event) => {
  await kv.hset("user1", { loggedIn: true });
  const loggedIn = await kv.hget("user1", "loggedIn");
  return { loggedIn };
});

api/logout

Route updates the loggedIn status to false and returns this value:

import { kv } from "@vercel/kv";

export default defineEventHandler(async (event) => {
  await kv.hset("user1", { loggedIn: false });
  const loggedIn = await kv.hget("user1", "loggedIn");
  return { loggedIn };
});

Layout

We have a single layout that uses one header component:

header.vue:

<template>
    <button v-if="loggedIn" @click="logout">Logout</button>
    <button v-else @click="login">Login</button>
</template>
<script setup lang="ts">
const loggedIn: Ref<boolean | undefined> = ref(false);
const { data } = await useFetch('/api/auth');
loggedIn.value = data?.value?.loggedIn;
const login = async () => {
    const response = await useFetch(
        "/api/login",
        {
            method: "PUT",
        }
    );
    loggedIn.value = response.data?.value?.loggedIn;
}
const logout = async () => {
    const response = await useFetch(
        "/api/logout",
        {
            method: "PUT",
        }
    );
    loggedIn.value = response.data?.value?.loggedIn;
}
</script>

This component simply renders a ‘Login’ button if the user isn’t logged in and a ‘Logout’ button if the user is logged in. It includes click event handlers for each button, which call their respective API routes.

Layout:

<template>
    <Header />
    <main class="main">
        <slot />
    </main>
</template>

Startup

Start the example project with:

git clone git@github.com:RisingStack/nuxt3-caching-with-auth.git
cd nuxt3-caching-with-auth
pnpm install

Create env file based on .env.example and start the app:

pnpm dev

User-specific data caching

If we examine our pages that should be cached with the current setup, we can observe that after logging in, upon page reload, the ‘Login’ button is still visible.

SWR without TTL

The button label only updates when the response changes.

SWR with TTL

The button label only updates when the TTL expires.

ISR without TTL

The button label isn’t updated as ISR without TTL means the page is cached permanently.

ISR with TTL

The button label only updates when the TTL expires.

SSR

When examining the SSR page, it functions as expected: upon the initial page load, the ‘Login’ button is visible. After logging in and reloading the page, the ‘Logout’ button is displayed.

What is causing this? The issue stems from both the SWR and ISR rendering modes caching the server-generated HTML response for the page. This implies that despite changes in the value provided by the API response, stale data persists in the browser until the TTL expires or the response changes, depending on the rendering mode.

Solution

To prevent caching of specific parts of the layout, page, or component, we can wrap them in the ClientOnly component provided by Nuxt. This ensures that the particular slot is rendered only on the client side.

Let’s modify the default layout:

<template>
    <Client-Only>
        <Header />
    </Client-Only>
    <main class="main">
        <slot />
    </main>
</template>

In addition, we need to modify the header component as useFetch used client-side does not fetch the data until hydration completes (docs):

header.vue:

[...]
<script setup lang="ts">
const loggedIn: Ref<boolean | undefined> = ref(false);
const { data } = await useFetch('/api/auth');
watch(() => data?.value?.loggedIn, () => {
    if (data.value) {
        loggedIn.value = data.value?.loggedIn
    }
})
[...]
</script>

This way, we are watching for changes in the response and are updating values of the loggedIn variable when they become available.

Upon checking the behavior now, it works as expected: any page reload after updating the user’s logged-in status will render the correct values.

SWR without TTL

The button label is up to date after a reload. The ‘Time in server-rendered HTML’ only updates when the response changes.

SWR with TTL

The button label is up to date after a reload. The ‘Time in server-rendered HTML’ only updates when the TTL expires.

ISR without TTL

The button label is up to date after a reload. However, the ‘Time in server-rendered HTML’ isn’t updated, as ISR without TTL means the page is cached permanently.

ISR with TTL

The button label is up to date after a reload. However, the ‘Time in server-rendered HTML’ only updates when TTL expires.

Deploying the App to Vercel

After importing the project to Vercel and configuring the Vercel KV storage, deployment becomes a matter of a single click (refer to the deployment information for more details).

It’s crucial to note that the SWR rendering mode only works with edge functions, while ISR functions exclusively with serverless functions. This distinction is not clearly documented — Vercel’s documentation typically encourages the use of ISR only, without acknowledging that it doesn’t support revalidation based on response changes. Consequently, we’ve raised a service ticket for this issue and are in communication with the Vercel team.

To enable edge functions, set the environment variable NITRO_PRESET=vercel-edge. Serverless functions are the default for deploying Nuxt projects to Vercel, so no additional configuration is required.

Deploying the App to Netlify

Initially, we also planned to use Netlify for this app. However, we soon discovered that the rendering modes in Nuxt 3, which provide caching, weren’t functioning correctly on Netlify. Regardless of the configuration we employed, some rendering modes didn’t work as expected (for more details, refer to the forum topic we opened on this issue).

Following discussions with the Netlify team, they redirected us back to Nuxt for resolution. As a result, we’ve opened an issue on the Nuxt GitHub repository to address this matter.

Conclusion

Opting for a rendering mode that facilitates caching is a great strategy to achieve faster load times and reduce server costs. This approach remains effective even when dealing with data on the page that requires regular updates or is user-specific. To address such scenarios, consider encapsulating the relevant components within the <ClientOnly> component provided by Nuxt.

For seamless one-click deployments, Vercel is a preferable choice, especially at the moment. This is due to the current issue where rendering modes supporting caching do not function correctly on Netlify. As the landscape evolves, it’s advisable to stay updated on platform-specific capabilities and limitations for the optimal deployment of your Nuxt app.

Getting Started With Elixir in 2024

We’ve already covered why Elixir and Phoenix are worth a try, but making the switch can be tricky. Elixir is a world apart from the JavaScript ecosystem, but we’re here to offer you a familiar reference point as you dive in. To do this, we’re crafting a series of articles that explain Elixir using JavaScript lingo. So, without further ado, let’s kick things off by diving into what Elixir is, how to get it up and running, and to wrap things up, we’ll show you how to create a “Hello, world!” application in a few different ways.

What is Elixir?

Elixir is a dynamic, functional programming language. This should not be so strange, as JavaScript is also dynamic and provides some functional aspects, like Array.prototype.map / filter / reduce and friends. In recent years, JavaScript has also moved away from APIs that mutate data and started to embrace a more immutable paradigm, where methods return new updated values, instead of overwriting the object they were called on.

Elixir runs on the Erlang Virtual Machine (called BEAM), which is somewhat analogous to how V8 works for Node.js. You might know that Erlang is also a language in its own right, so what’s the deal? Just as with V8, which supports different languages like TypeScript, ClojureScript, Scala.js, and CoffeeScript (RIP), BEAM has its unique ecosystem. However, while TypeScript and others compile to JavaScript, both Elixir and Erlang compile into BEAM bytecode. This setup is more similar to JVM languages like Java, Scala, Clojure, and Kotlin. If you’re not familar with these, think of it as when JavaScript is parsed, it would be compiled into wasm instrucutions. In that case, JS would also be a wasm target like all other languages that have a wasm compiler: C++, Rust, Go etc.

However, the BEAM is not like any other VM. It would be beyond the scope of this post to delve into the fault tolerance provided by this technology, but as you write your code in Elixir, you’ll notice that when something breaks, the effect is similar to that in JavaScript: only the part of the application where you had the error breaks, and the rest continues to function. But you’ll probably find it much more difficult to crash an entire Elixir application than a Node.js backend. The reason behind this is Elixir’s concurrency model, which is based on lightweight BEAM processes functioning as actors, in line with the actor model. This makes reasoning about your code in Elixir a lot easier than in Node.js. Most tasks run in separate processes, so many operations can be synchronous. It’s akin to using worker threads for every request your server handles, but much more lightweight and easier to manage. However, unlike worker threads, or threads in general, BEAM processes don’t share memory, making it very difficult – if not virtually impossible – to encounter race conditions. That’s one of the reasons why Elixir has gained popularity, particularly for developing robust and scalable web applications using the Phoenix framework.

While we’re on the topic, let’s touch on OTP. When installing Elixir, you’ll also need to install Erlang, ensuring that their versions are compatible. Most of the time, however, the Erlang version will be referred to as the Erlang/OTP version or simply the OTP version. OTP comprises a set of libraries usable in both Erlang and Elixir. But it’s not your typical lodash or express. It includes abstractions over BEAM processes, an Application concept, a method for communication between BEAM nodes, a Redis-like distributed term storage called ETS, and Mnesia, which is AN ACTUAL BUILT-IN DATABASE similar to MongoDB.

And let’s pause for a moment to talk about communicating between BEAM nodes. Essentially, you can start Elixir apps on different machines, link them together, and then call functions from one node on another. There’s no need for HTTP, messaging queues, or REST APIs. You simply call the function on one machine and receive the result from another.

This is why it was so straightforward for Chris McCord to implement Fly.io’s FLAME serverless/lambda-like service for Elixir. FLAME’s spiritual predecessor, Modal was developed for machine learning in Python, but it took an entire company and years to complete.

How to install Elixir?

Just like with Node.js, you there are multiple ways to install Elixir. You can use your OS’s package manager, run it with Docker, or download prebuilt binaries. However, you’ll probably want to be able to control which version of Elixir you’re using, so the best is to use a version manager. In our experience, it’s also the easiest way.

What’s up with Elixir and Erlang compatibility?

When working with Elixir and Erlang, it’s generally recommended to use compatible versions of both to avoid potential issues. The compatibility between Elixir and Erlang versions is crucial, as certain features or enhancements in Elixir may rely on specific Erlang/OTP releases, given their shared execution environment on the BEAM virtual machine. If you install incompatible versions, you might encounter issues such as:

  • Functionality Breakage: Certain Elixir features may depend on Erlang/OTP features introduced in specific versions.
  • Performance Issues: Newer versions of Erlang/OTP often come with performance improvements and bug fixes.
  • Potential Bugs: Running Elixir on an incompatible Erlang version may lead to unexpected behavior, errors, or even crashes due to mismatches in the underlying runtime.

Checking the compatibility matrix in the Elixir documentation is a recommended approach. For optimal performance, use the Erlang version against which Elixir was compiled.

Which version manager to use?

You might be tempted to go the Node.js way and look for a language specific version manager. They exist, namely kiex for Elixir and kerl for Erlnag. However, we found the easiest is to use asdf instead, which is a multi-language version manager that supports multiple languages, including Elixir, Erlang, Node.js, Ruby, Python, and more. The added benefit of asdf comes out when you work on projects that involve multiple languages – in contrast, nvmkiex, and kerl are specifically designed for their respective languages.

To install asdf, follow the instructions here.

Aftrer you install asdf, however, you’re not ready to start downloading runtimes yet. Actually, asdf is more like a backend for multiple version managers that are called plugins in asdf parlance. In the following, we’ll look at how to add language plugins, install Erlang and Elixir, then set the versions to be used.

Install Erlang and Elixir with asdf

Let’s add Erlang first:

asdf plugin-add erlang https://github.com/asdf-vm/asdf-erlang.git

Then Elixir:

asdf plugin-add elixir https://github.com/asdf-vm/asdf-elixir.git

Now we need to check the available Elixir versions first with asdf list-all elixir.

Notice the otp-XX suffix at the end of version names. That’s how we know against which Erlang version was the specific runtime compiled. Pick one you like, in our case, let’s go with the current latest, OTP 26 in our case.

Let’s take a look at the available Erlang versions too.

At the time of writing, 26.2.1 is the latest, so we’re going to install that.

asdf install erlang 26.2.1

And now, we’re ready to install the latest Elixir version.

asdf install elixir 1.16.0-otp-26

To verify the install, we just need to start the Erlang REPL

erl

Let’s verify this install too.

elixir -v

Local and Global versions

Unlike nvmasdf makes it seamless to use project local versions. With nvm you create a .nvmrc file and whenever you enter the project root directory you need to run nvm use to switch to the proper Node version or alias the default version as… well… default. On the other hand, with asdf you can set project local and system-wide global versions.

  1. Global
asdf global erlang 26.2.1

asdf global elixir 1.16.0-otp-26
  1. Local

In you projects root directory run the following command.

asdf local elixir 1.16.0-otp-26

asdf local erlang 26.2.1

This will create a .tool-versions file with the defined versions.

Now every time you cd into that directory, asdf will automatically set the runtime versions to the one you need for the given project.

You can verify the version in use by starting the Erlang REPL and running elixir -v

erl
Erlang/OTP 26 [erts-14.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Eshell V14.2.1 (press Ctrl+G to abort, type help(). for help)
1> halt().

elixir -v
Erlang/OTP 26 [erts-14.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Elixir 1.16.0 (compiled with Erlang/OTP 26)

Notice the . at the end of the halt(). call in the Erlang REPL. You can exit with a double Ctrl+C too, but it’s just more elegant.

Hello World in Elixir

First, launch a the IEx (Interactive Elixir) REPL in the terminal:

iex
iex(1)> IO.puts "Hello, world!"
Hello, world!
:ok
iex(2)>

When calling a function, wrapping the arguments in () is optional. This can be very convenient when you’re just playing around in the REPL.

So far so good. But when you started out with Node, you probably wrote and index.js file with console.log in it and ran it with node. For me it was definitely needed to feel like a big boy.

Let’s do so by creating a file called hello_world.exs.

Once it’s saved, we’re ready to execute it.

elixir hello_world.exs

Wait, what did just happen? I told you that Elixir is a compiled language, yet we ran our Hello, world! just like you do with a script. Well, you don’t necessarily need to save the binaries to a file, do you? When you run some Elixir code with the elixir command, it get’s compiled, but only held in memory, which can be useful for setup scripts, mix tasks and the likes. By convention, .exs files are used this sript-like way and ex files are compiled and serialized into files.

All right then, how do we compile Elixir programs properly? Now that’s a bit more complex, as most of the time, you will use releases. But for now let’s do it the way you’d create CLI programs, even though you’ll most likley never do so. It’s only to get a some sort of fulfillment.

Our first – very simple – project

Let’s create or first Elixir project with the help of mix, which is somewhat similar to npm: you use it to download packages, build your projects or run them in development mode. Let us know if you’d like a post on comparing  npm and  package.json with  mix and  mix.exs.

Time to get back to your terminal of choice and run:

mix new elixir_hello_world

It creates a simple project library structure like.

elixir_hello_world
├── README.md
├── lib
│   └── elixir_hello_world.ex
├── mix.exs
└── test
    ├── elixir_hello_world_test.exs
    └── test_helper.exs

Let’s open lib/elixir_hello_world.ex it should look something like this:

defmodule ElixirHelloWorld do
 @moduledoc """
 Documentation for `ElixirHelloWorld`.
 """

 @doc """
 Hello world.

 ## Examples
 
  iex> ElixirHelloWorld.hello()
  :world

 """ 
 def hello do
  :world
 end
end

In it’s current from it simply returns the atom :world. However, that’s not useful for us now, as we don’t care about the return value, just want to print something to stdout. Let’s replace the return value with our previous IO.puts call.

def hello doc
 IO.puts("Hello, world!")
end

Now we can call our hello function using mix by specifying the app name, module name and function, still without prior compilation.

mix run -e ElixirHelloWorld.hello

Compiling 1 file (.ex)
Hello, world!

We can also load our project in the REPL:

iex -S mix
Erlang/OTP 26 [erts-14.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Interactive Elixir (1.16.0) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> ElixirHelloWorld.hello
Hello, world!
:ok
iex(2)>

Now let’s tell mix that this is our main module. Time to open our mix.exs file. Find the part that says def project do, and add escript: [main_module: ElixirHelloWorld], the following to the list within the do ... end block. It should look like this:

def project do
 [
  app: :elixir_hello_world,
  version: "0.1.0",
  elixir: "~> 1.16",
  start_permanent: Mix.env() == :prod,
  deps: deps(),
  escript: [main_module: ElixirHelloWorld],
 ]
end

We also need to rename our hello function.

def main(_arg) do
  IO.puts("Hello, world!")
end

Prepending our argument name with an underscore tells the compiler that we don’t care about it’s value, in turn we don’t get unused variable warnings.

Finally, we’re ready to compile our first Elixir project!

mix escript.buildCompiling 1 file (.ex)
Generated escript elixir_hello_world with MIX_ENV=dev
./elixir_hello_worldHello, world!

Where to go from here?

That’s the end of our intro to Elixir. We’re planning to write more posts like this, where we try to explain the language we grew to love in JavaScript terms. In the meantime, we recommend exploring the official documentation of Elixir. And we do mean it, as it is probably the best official documentation and tutorial of a language we’ve ever seen, so it should definitely be your starting point. If you find the docs from Google though, make sure you switch to 1.16.0, using the dropdown menu in the upper left corner, as it points to the documentation of older versions.

Update Elixir & Phoenix & LiveView to the Latest Version

This article serves as your one-stop resource for all the necessary information on updating these key components of the Elixir ecosystem. You’ll also find the recent changes and enhancements that have been made to Elixir, Phoenix, and LiveView – changelogs included.

Whether you’re new to this tech stack, or just need quick info to get the latest version, we’ve got you covered.

Updating Elixir to v1.16.0

On Linux / MacOS

At first glance, when reading the Elixir install guide, you might be tempted to think it suggests using homebrew for installing Elixir on MacOS. However, it even states that you might want to look at the version managers listed below. So what gives? If you want to use a runtime/compiler as a developer, always use a version manager, as you will need an easy way to both lock old versions, update to new ones, and switch between different versions. On the other hand, if you just want to use said runtime/compiler, e.g., to run CLI tools with Node.js, or install programs with go install or cargo install, you’ll be fine with any OS-specific package manager. In the case of Elixir, however, we don’t know any good reason why you wouldn’t stick to a version manager, as people seldom use it to distribute CLI tools, save for Elixir development-specific ones.

Now, the only question that remains is which one to use. kiex + kerl or asdf? Let us save you a couple of hours of headache and suggest using asdf. It can handle both, and you can even install project specific versions and it will switch between them automatically. .

Run elixir --version to find out which version you have already installed.

Getting started with asdf: https://asdf-vm.com/guide/getting-started.html

GitHub repo: https://github.com/asdf-vm/asdf

First, add the plugins for Elixir and Erlang with the following commands:

asdf plugin-add erlang
asdf plugin-add elixir

If you already have the plugins installed, make sure they are up to date with these commands:

asdf update
asdf plugin-update --all

Before continuing, note that you need compatible Erlang and Elixir versions otherwise, you might find yourself in a pickle.

You can check the compatibility table in the official documentation: https://elixir-lang.org/docs

You can check the latest available versions of Elixir and Erlang:

$ asdf latest elixir
1.16.0-otp-26

$ asdf latest erlang
26.2.1

If the latest versions are compatible with each other, you can simply update both to the latest version:

asdf install erlang latest
asdf install elixir latest

Otherwise, list all the available versions, then pick and choose what you need.

asdf list all erlang
asdf list all elixir

When you list Elixir, you’ll see the OTP version it was compiled against as an -otp-XX suffix, eg:

1.15.7
1.15.7-otp-24
1.15.7-otp-25
1.15.7-otp-26
1.16.0
1.16.0-otp-24
1.16.0-otp-25
1.16.0-otp-26
1.16.0-rc.0
1.16.0-rc.0-otp-24
1.16.0-rc.0-otp-25
1.16.0-rc.0-otp-26
1.16.0-rc.1
1.16.0-rc.1-otp-24
1.16.0-rc.1-otp-25
1.16.0-rc.1-otp-26
main
main-otp-22
main-otp-23
main-otp-24
main-otp-25
main-otp-26

Once you selected the versions you need, you’re ready to specify which to install

asdf install erlang VERSIONNUMBER
asdf install elixir VERSIONNUMBER

We have a more detailed install guide coming very soon that includes a guide on how to install specific versions per project. Make sure to subscribe to our newsletter to be among the first to get it!

On Windows

You can do it the easy way or the hard way.

The easy way: using chocolatey

Install chocolatey, then run 

choco install elixir

This will install Erlang as a dependency too.

The hard way: with the installer

We’d only go that way if you already have Erlang installed on your Windows machine, and you want to stick to your current install. Either way:

  1. If you haven’t done so already, download and install the latest version of Erlang: https://www.erlang.org/downloads.html
    1. If you want to check which version of Erlang you have (if any), run erl in the terminal.
  2. After that, download the Elixir installer that is compatible with the version of Erlang that you have installed:
    1. https://github.com/elixir-lang/elixir/releases/download/v1.16.0/elixir-otp-26.exe
    2. https://github.com/elixir-lang/elixir/releases/download/v1.16.0/elixir-otp-25.exe
    3. https://github.com/elixir-lang/elixir/releases/download/v1.16.0/elixir-otp-24.exe

Note: you need to have Erlang v24.0 or later installed for Elixir v.1.16.

If you already have a previous version of Elixir installed and just want to update, run the installer as usual and check the “replace my current version” when the installer asks.

The best way: Using WSL

Unless you really, really, really have to install Elixir on a Windows host, we suggest using it in WSL instead. Ports get automatically forwarded to the host machine, so you won’t have to do the rain dance to reach your local development server. You’ll be able to use asdf, and in general, you’ll have a nicer time. The only downside, in our experience, is that the LSP autosuggest is a bit slower that way than if the whole dev env was run on the host machine. In that case, Windows – Linux dual boot still looks preferable.

Elixir changelog of the latest minor version (1.16.0)

  • Compiler Diagnostics: Enhanced diagnostics with code snippets and ANSI coloring for errors like syntax errors, mismatched delimiters, and unclosed delimiters.
  • Revamped Documentation: Learning materials moved to the language repository, enabling full-text search across API references and tutorials, with ExDoc autolinking to relevant documentation.
  • Introduction of Cheatsheets and Diagrams: Starting with the Enum module and incorporating Mermaid.js diagrams in docs like GenServer and Supervisor.
  • Anti-patterns Reference: Inclusion of anti-patterns categorized into code-related, design-related, process-related, and meta-programming, providing guidance for developers.
  • Other Notable Changes: Addition of String.replace_invalid/2, a :limit option in Task.yield_many/2, and improvements in binary pattern matching.

Building a Phoenix application with the latest version: 1.7.11

Before you proceed, make sure you have both Elixir and Erlang installed and both are up-to-date and the versions you have are compatible with each other.

The first step is to get the Hex package manager:

mix local.hex

Don’t worry about Mix, it comes with Elixir – if you have Elixir installed then you have Mix as well.

To install the Phoenix project generator,  run the following command:

mix archive.install hex phx_new

You’re ready to generate your project:

mix phx_new <your_project_name>

Make sure you use a snake_case-d name, which also means that all letters should be lowercase! 

The command above sets up the project with a basic landing page as well as the server, router, migrations, db connection, and anything you need to get started. Speaking of database connection: PostgreSQL is the default one, but you can also switch to MySQL, MSSQL, or SQLite3 with the --database flag.

If you aren’t planning on using databases with your app, use the --no-ecto flag when creating a new Phoenix app. Ecto is an Elixir package that – among many other things – helps establishing a database connection. 

Updating Phoenix

If you just want to make sure that your app uses the latest version of Phoenix, you can simply update the version number in your mix.exs file.

For patches, you shouldn’t run into any issues, but for minor updates, refer to the official Phoenix documentation to make sure you can update without a hitch: https://github.com/phoenixframework/phoenix/blob/main/CHANGELOG.md

Phoenix changelog of the latest minor version (v1.7)

  • Verified Routes: Introduces Phoenix.VerifiedRoutes for compile-time verified route generation using ~p sigil, enhancing reliability and reducing runtime errors.
  • phx.new Revamp: The phx.new application generator has been improved to rely on function components for both Controller and LiveView rendering, ultimately simplifying the rendering stack of Phoenix applications and providing better reuse. The revamp also introduces improvements  in the application generator, focusing on function components and Tailwind CSS for styling. This simplifies rendering and offers better style management.
  • JavaScript Client Enhancements: New options for long poll fallback and debugging, improving WebSocket connectivity and logging.
  • Various Bug Fixes and Enhancements: Across different versions (1.7.1 to 1.7.11), addressing issues in code reloader, controller, channel test, and more. Enhancements include dynamic port support for Endpoint.url/0, updated socket drainer configuration, and support for static resources with fragments in ~p. .

Installing and updating LiveView

If you’re using the latest version of phx.new and Phoenix, you’re in luck, because it has built-in support for LiveView apps. To get started with LiveView, you just simply get started with Phoenix:

mix phx.new <your_project_name>

On older versions you might need to add the --live flag, though it is the default since v1.6.

Updating LiveView is as easy as updating the dependencies in your app’s mix.exs file to the latest version, and then run mix deps.get.

The current latest version is 0.20.3.

Full changelog: https://hexdocs.pm/phoenix_live_view/changelog.html#0-20-3-2024-01-02

LiveView changelog of the latest minor version (0.20.0)

  • Deprecations: The shift from older syntax and functions to new ones is crucial. This includes:
    • Deprecating the ~L sigil in favor of ~H.
    • Moving from preload/1 in LiveComponent to update_many/1.
    • Transitioning from live_component/2-3 to <.live_component />.
    • Replacing live_patch with <.link patch={…} />.
    • Changing live_redirect to <.link navigate={…} />.
    • Switching from live_title_tag to <.live_title />.
  • Backwards Incompatible Changes: Removal of deprecated functions like render_block/2, live_img_preview/2, and live_file_input/2 in favor of new implementations (render_slot/2, <.live_img_preview />, <.live_file_input />). These changes can break existing code that hasn’t been updated.

You can find the full changelog here: https://hexdocs.pm/phoenix_live_view/changelog.html#0-20-0-2023-09-22

Facing challenges in your Elixir development? Contact our team for professional help and get your project moving smoothly!

Comparing Nuxt 3 Rendering Modes: SWR, ISR, SSR, SSG, SPA

Update: to enhance the clarity of the Nuxt 3 documentation, we have opened a pull request (PR) that has already been merged. Now, the functionality of ISR/SWR rendering modes is better explained.

The backstory

We were developing a listing site using Nuxt 3 and aimed to optimize page load times while maintaining SEO benefits by choosing the right rendering mode for each page. Upon investigating this, we found that documentation was limited, particularly for newer and more complex rendering modes like ISR. This scarcity was evident in the lack of specific technical details for functionality and testing.

Moreover, various rendering modes are referred to differently across knowledge resources, and there are notable differences in implementation among providers such as Vercel or Netlify. This led us to compile the information into the following article, which offers a clear, conceptual explanation and technical insights on setting up different rendering modes in Nuxt 3.

Knowledge prerequisites: Basic understanding of Nuxt.

Rendering modes

Project setup

The project consists of 7 pages, each displaying the current time and an HTML response from the same route. Specifically, the route /api/hello returns a JSON response with the current time, and each page features a different available rendering mode enabled.

The pages differ only in their title that refers to the rendering mode used for generating them. In the case of the simple SPA for example:

<template>
    <div>
        <p>{{ pageType }} page</p>
        <pre>Time after hydration: {{ new Date().toUTCString() }} </pre>
        <pre>Time in server rendered HTML: {{ data }}</pre>
        <NuxtLink to="/">Home</NuxtLink>
    </div>
</template>
<script setup lang="ts">
const pageType = "SPA"; // value differs for each route
const { data } = await useFetch('/api/hello')
</script>

To make it visible when the page was rendered, we showcase 2 timestamps on the site:

  1. One we get from the API response, to see when was the page rendered by the server:
<template>
[...]
        <pre>Time in server rendered HTML: {{ data }}</pre>
[...]
</template>

<script setup lang="ts">
const { data } = await useFetch('/api/hello')
</script>
  1. And one in the browser:
<pre>Time after hydration: {{ new Date().toUTCString() }} </pre>

We utilize these two timestamps to demonstrate the functionality of each rendering mode, focusing on the hydration process. In case you’re new to SSR frameworks: First, we send the browser a full-fledged HTML version of the initial state of our site. Then it get’s hydrated, meaning Vue takes over, builds whatever it needs, runs client-side JavaScript if necessary and attaches itself to the existing DOM elements. From here on, everything works the same as with any other SPA. In our scenario, this implies that the current first <pre> element will always display the timestamp of the time the page got rendered by the browser, while the second <pre> element showcases the time Vue got the response from the API, thus when the HTML got rendered on the server.

Our API route is as simple as this:

export default defineEventHandler((event) => {
  return new Date().toUTCString();
});

Rendering modes are set up in nuxt.config:

export default defineNuxtConfig({
  devtools: { enabled: true },
  ssr: true,
  routeRules: {
    "/isr_ttl": { isr: 60 },
    "/isr_no_ttl": { isr: true },
    "/swr_ttl": { swr: 60 },
    "/swr_no_ttl": { swr: true },
    "/ssg": { prerender: true },
    "/spa": { ssr: false },
  },
});

Startup

Start the example project with:

git clone git@github.com:RisingStack/nuxt3-rendering-modes.git
cd nuxt3-rendering-modes
pnpm install
pnpm dev

SPA

Single Page Application (also called Client Side Rendering).

HTML elements are generated after the browser downloads and parses all the JavaScript code containing the instructions to create the current interface.

We use the route /spa to illustrate how this rendering mode works:

DataValue
Time in server rendered HTMLHTML response is blank
Time in API responseTue, 16 Jan 2024 09:47:10 GMT
Time after hydrationTue, 16 Jan 2024 09:47:10 GMT

As we can see in the table, the HTML response is blank, and the “Time after hydration” matches the “Time in API response”. This occurs because the API request is made client-side. On subsequent requests or page reloads, the HTML response will always be blank, and the time will change with each request. However, the browser-rendered value and the API response value will consistently be the same.

To enable this mode, set up a route rule in nuxt.config as follows:

export default defineNuxtConfig({
  routeRules: {
    "/spa": { ssr: false },
  },
});

SSR

Server Side Rendering (also called Universal Rendering).

The Nuxt server generates HTML on demand and delivers a fully rendered HTML page to the browser.

We use the route /ssr to illustrate the behaviour of this rendering mode:

DataValue
Time in server rendered HTMLTue, 16 Jan 2024 09:47:45 GMT
Time in API responseTue, 16 Jan 2024 09:47:45 GMT
Time after hydrationTue, 16 Jan 2024 09:47:48 GMT

In this case, the “Time after hydration” might slightly differ from the “Time in API response” since the API response is generated beforehand. However, the timestamps will be very close to each other because the HTML generation occurs on demand and is not cached. This behavior will remain consistent across subsequent requests or page reloads.

To enable this mode, enable SSR in nuxt.config as follows:

export default defineNuxtConfig({
  ssr: true
});

SSG

Static Site Generation

The page is generated at build time, served to the browser, and will not be regenerated again until the next build.

The route /ssg demonstrates SSG behavior:

DataValue
Time in server rendered HTMLTue, 16 Jan 2024 10:00:41 GMT
Time in API responseTue, 16 Jan 2024 10:00:41 GMT
Time after hydrationTue, 16 Jan 2024 10:09:09 GMT

In the table mentioned above, there is a noticeable time difference between the time after hydration and other timestamps. This is because, in SSG mode, HTML is generated during build time and remains unchanged afterward. This behavior will persist across subsequent requests or page reloads.

To enable this mode, set up a route rule in nuxt.config as follows:

export default defineNuxtConfig({
  routeRules: {
    "/ssg": { prerender: true },
  },
});

SWR

Stale While Revalidate

This mode employs a technique called stale-while-revalidate, which enables the server to provide stale data while simultaneously revalidating it in the background. The server generates an HTML response on demand, which is then cached. When deployed, the caching specifics can vary depending on the provider (eg. Vercel, Netlify, etc.), and information about where the cache is stored is usually not disclosed. There are two primary settings for caching:

  1. No TTL (Time To Live): This means the response is cached until there is a change in the content.
  2. TTL Set: This implies that the response is cached until the set TTL expires.

Nuxt saves the API response that was used for generating the first version of the page. Then upon all subsequent requests, only the API gets called, until the response changes. When a change is detected during a request – with no TTL set – or when the TTL expires, the server returns the stale response and generates new HTML in the background, which will be served for the next request.

SWR without TTL

To observe the behavior of the SWR mode without a TTL set, you can take a look at the /swr_no_ttl route:

DataValue – first requestValue – second requestValue – third request
Time in server rendered HTMLTue, 16 Jan 2024 09:48:55 GMTTue, 16 Jan 2024 09:48:55 GMTTue, 16 Jan 2024 09:49:02 GMT
Time in API responseTue, 16 Jan 2024 09:48:55 GMTTue, 16 Jan 2024 09:48:55 GMTTue, 16 Jan 2024 09:49:02 GMT
Time after hydrationTue, 16 Jan 2024 09:48:58 GMTTue, 16 Jan 2024 09:49:03 GMTTue, 16 Jan 2024 09:49:10 GMT

Let’s dissect the above table a bit.

In the first column, the behavior is similar to that observed with SSR, as the “Time after hydration” slightly differs from the “Time provided in API response”. In the second column, it appears the user waited around 5 seconds before reloading the page. The content is served from the cache, with only the time after hydration changing. However, this action triggers the regeneration of the page in the background due to the change in the API response since the first page load. As a result, a new version of the page is obtained upon the third request. To understand this, compare the “Time after hydration” in the second column with the “Time in server rendered HTML” in the third column. The difference is only about 1 second, indicating that the server’s rendering of the third request occurred almost concurrently with the serving of the second request.

To enable this mode, set up a route rule in nuxt.config as follows:

export default defineNuxtConfig({
  routeRules: {
    "/swr_no_ttl": { swr: true },
  },
})
SWR with TTL

This rendering mode is set up on /swr_ttl route:

DataValue – first requestValue – second requestValue – first request after TTL of 60 seconds passedValue – second request after TTL of 60 seconds passed
Time in server rendered HTMLTue, 16 Jan 2024 09:49:52 GMTTue, 16 Jan 2024 09:49:52 GMTTue, 16 Jan 2024 09:49:52 GMTTue, 16 Jan 2024 09:50:58 GMT
Time in API responseTue, 16 Jan 2024 09:49:52 GMTTue, 16 Jan 2024 09:49:52 GMTTue, 16 Jan 2024 09:49:52 GMTTue, 16 Jan 2024 09:50:58 GMT
Time after hydrationTue, 16 Jan 2024 09:49:55 GMTTue, 16 Jan 2024 09:50:00 GMTTue, 16 Jan 2024 09:51:00 GMTTue, 16 Jan 2024 09:51:06 GMT

In this scenario, the values of the first request in the /swr_ttl route are again similar to those observed in SSR mode, with only the time after hydration differing slightly from the other values. For the second and subsequent requests, until the TTL of 60 seconds expires, the “Time in API response” row retains the same timestamp as the first request. After the TTL expires (as shown in the third column), the time in the API response is still stale. However, in the fourth column, a new timestamp appears in the “Time in API response” row, indicating that the content has been updated post-TTL expiry.

To enable this mode, set up a route rule in nuxt.config as following:

export default defineNuxtConfig({
  routeRules: {
    "/swr_ttl": { swr: 60 },
  },
})

ISR

Incremental Static Regeneration (also called Hybrid Mode)

This rendering mode operates similarly to SWR (Stale-While-Revalidate), with the primary distinction being that the response is cached on a CDN (Content Delivery Network). There are two potential settings for caching:

  1. No TTL (Time To Live): This implies that the response is cached permanently.
  2. TTL Set: In this case, the response is cached until the TTL expires.

Note: ISR in Nuxt 3 differs significantly from ISR in Next.js in terms of HTML generation. In Nuxt 3, ISR generates HTML on demand, while in Next.js, ISR typically generates HTML during the build time by default.

ISR without TTL

This mode is available on the /isr_no_ttl route:

DataValue – first requestValue – second requestValue – third request
Time in server rendered HTMLTue, 16 Jan 2024 09:52:54 GMTTue, 16 Jan 2024 09:52:54 GMTTue, 16 Jan 2024 09:52:54 GMT
Time in API responseTue, 16 Jan 2024 09:52:54 GMTTue, 16 Jan 2024 09:52:54 GMTTue, 16 Jan 2024 09:52:54 GMT
Time after hydrationTue, 16 Jan 2024 09:52:56 GMTTue, 16 Jan 2024 09:53:03 GMTTue, 16 Jan 2024 09:53:11 GMT

In the table and screencast provided, it’s evident that the value in the “Time in API response” row remains unchanged, even after 60 seconds have elapsed, which is typically the default TTL for Vercel. This observation aligns with the behavior of ISR without TTL in Nuxt 3, where the content is cached permanently.

To enable this mode, set up a route rule in nuxt.config as follows:

export default defineNuxtConfig({
  routeRules: {
    "/isr_no_ttl": { isr: true },
  },
})
ISR with TTL

The route /isr_ttl demonstrates ISR behaviour without TTL:

DataValue – first requestValue – second requestValue – first request after TTL of 60 seconds passedValue – second request after TTL of 60 seconds passed
Time in server rendered HTMLTue, 16 Jan 2024 10:01:21 GMTTue, 16 Jan 2024 10:01:21 GMTTue, 16 Jan 2024 10:01:21 GMTTue, 16 Jan 2024 10:02:24 GMT
Time in API responseTue, 16 Jan 2024 10:01:21 GMTTue, 16 Jan 2024 10:01:21 GMTTue, 16 Jan 2024 10:01:21 GMTTue, 16 Jan 2024 10:02:24 GMT
Time after hydrationTue, 16 Jan 2024 10:01:24 GMTTue, 16 Jan 2024 10:01:28 GMTTue, 16 Jan 2024 10:02:25 GMTTue, 16 Jan 2024 10:02:32 GMT

For the first request on the /isr_ttl route, the observed values are similar to the SSR behavior behavior, with only the time after hydration showing a slight difference. During the second and subsequent requests, until the TTL of 60 seconds passes, the “Time in API response” row retains the same timestamp as in the first request. After the TTL expires (as shown in the third column), the time in the API response remains stale. It’s only in the fourth column that a new timestamp appears in the “Time in API response” row, indicating an update post-TTL expiry.

To enable this mode, set up a route rule in nuxt.config as follows:

export default defineNuxtConfig({
  routeRules: {
    "/isr_ttl": { isr: 60 },
  },
})

Note that all the above mentioned rendering modes, except for ISR, can be easily tested in a local environment by building and previewing the app. ISR, however, relies on a CDN network for its functionality, which means it requires a CDN for proper testing. For example, deploying to Vercel would be necessary to test ISR effectively.

Conclusion

Now that we’ve explored how different rendering modes work technically, let’s discuss their pros and cons. Here’s our take:

If you’re not concerned about SEO, social media, or the initial load time, opt for Single Page Application (SPA). But if the above three matter, you should consider one of the server-generated modes.

Static Site Generation (SSG) is the easiest if your content doesn’t change frequently. Server-Side Rendering (SSR) is better for real-time updates, even though it takes a bit longer to load compared to modes with caching like Incremental Static Regeneration (ISR) or Stale-While-Revalidate (SWR).

Serving outdated data might sound scary at first, but in a lot of use cases you’ll find that it does not really matter – in those cases, ISR or SWR is the way to go. They provide caching, reducing server costs and load times. ISR loads faster than SWR, thanks to the CDN, although it doesn’t support HTML regeneration based on response updates.

Why Elixir? Phoenix & LiveView are unmatched for Modern Web Apps

Let’s face it: In the JavaScript world, we still don’t have a killer app. 

We’ve previously written about Redwood and Blitz, two technologies that seemed promising at the time, but they’re still not really there, and we do not see them taking the community by storm. (You can read about them here: RedwoodJS vs. BlitzJS: The Future of Fullstack JavaScript Meta-Frameworks)

On the other hand, Next.js has become the de-facto standard for full-stack development, but it’s still far from becoming a kind of “React on Rails”, and compared to other frameworks, a Next.js project is definitely not smooth sailing either.

In our meta-framework post, we briefly mentioned Elixir’s killer app, Phoenix, and LiveView. If you felt it was foreshadowing what was to come, you were right.

Because of Phoenix and LiveView

Phoenix is basically Rails for Elixir, but unlike Rails, it scales really well. And just like Rails, it comes with a powerful code generator: you simply define your data model, and a full-stack CRUD feature is generated for you with migrations, models, basic business logic, templates, components, and forms. It uses Ecto, which provides migrations, query builder model definitions, and validations for those models… and forms too! You can create forms based on your Ecto models, and Phoenix handles the form validation for you automatically, so you don’t need to repeat the same thing in multiple places. 

But Phoenix provides much more: Do you need authentication? Just run `phx gen.auth`, and you have everything from registration through login to email validation and forgotten password. You need to notify a subset of clients of events? Use Phoenix.Channel. Need to see who’s online? Phoenix.Presence tells you exactly that. And the list goes on.

To explain why LiveView is awesome, we’ll need to dig a bit deeper into why it’s superior to SPAs first. But long story short, the speed and simplicity provided by Phoenix and LiveView are just unimaginable after working on SPAs for almost a decade. How fast, you ask? Look at this video where a live twitter clone gets implemented in 17 minutes.

So just to answer my own question: Why Elixir? Because of Phoenix LiveView.

What’s wrong with the Web today?

We used to build backends for web pages. They were simple but weren’t really interactive.

Rails architecture

Then came mobile apps, and we loved the interactivity. We wanted to have the same in the browser, so we started building web Single-Page Apps (SPAs). 

SPA structure

While the architecture of a SPA does not seem that much different from a simple set of pages, the Client-Server separation made everything a lot more complicated. Previously, we had one system that simply generated HTML strings, and we added some JavaScript to it here and there. Now instead, we have a backend API and a frontend app that are essentially two different systems with their own states, their own validations, and their own storage (think LocalStorage and IndexedDB).

We started to not only deliver HTML with CSS and some JavaScript logic for DOM manipulation to the browser but also a whole framework with a complex application. Inevitably, load times became slower and slower as the amount of code we sent over the wire kept growing. This is not a problem for applications that we open and use throughout the day, like webmail clients, instant messaging platforms, or to-do apps, but we use these frameworks for literally everything. 

And why? Because we have to. 

Take, for example, a simple listing site, which sounds like a good target for a simple web page: it has to be SEO friendly, listings have to be loaded quickly, and at first glance, it seems pretty static. Still, it needs interactive filters, navigation, and loading animations, so even though the majority of the content could be easily generated on the server, we end up needing to write a full-fledged web app for that too. So we needed to figure out how to render JavaScript on the server side. Or at least we thought. 

Next structure

So we started to do just that and started using Next.js, Nuxt, and SvelteKit. But rendering JavaScript on the backend is ridiculously resource-heavy. While in the olden days, we simply needed to replace variables in an HTML string template, now we need to run JS on the server as a browser would so we can generate the same thing. So while we have the possibility to use SSR, we should only rely on it when we can easily cache the generated pages on a CDN. 

Well, if it changes so rarely, we can statically generate it, can’t we? But for now, most frameworks handle SSG in an “all or nothing” manner, so if the data behind one-page changes, we need to regenerate the whole site. So in those cases, we better rely on Incremental Static Regeneration (ISR), where the page only gets generated when it’s requested for the first time and gets cached for a given period of time. If the page gets accessed after its TTL has expired, the stale page gets served to the user, but a new version is generated in the background. So we’re juggling with SSR, SSG, and ISR, which makes it even more difficult to reason about our system. Mind you: this whole complexity is there so we can have some interactivity while forgoing the need to show our users a loading bar when they first navigate to our page.

Web development really got out of hand in the last decade.

LiveView to the rescue

The whole problem arose because our toolset is binary: A site is either interactive or simple. No middle ground, while the majority of the apps we build could do with some sparkle form validation, navigation, and filtering and could leave the other parts static. 

LiveView

The idea behind LiveView is fairly simple: You create templates that get rendered into HTML strings, then ship them with minuscule JavaScript that latches on to form controls and links. The JS lib also builds a WebSocket connection between the client and the server. Through that connection, interactions get streamed to the server. You update the state on the backend as you would do with react, and the diff between the old state and the new state gets sent to the frontend through the socket. Then the LiveView JS lib applies said diff to the DOM. 

And that’s it. When the user navigates to our page, LiveView replaces the variables in the template just like any other simple template engine would do instead of mimicking a browser on the server. It’s fast, interactive, and simple, just as we needed. And all this using ~1000 lines of JavaScript (88Kb without gzip and minification).

Why we’ll help you learn Elixir

So if you’re a startup or working on side projects and you build web apps, you should definitely start learning Elixir, Phoenix, and LiveView. It might take a couple of weeks to get productive, but the fact that with them, one person can achieve what 3-5 other engineers can do in the same amount of time with other tools starts to pay dividends quickly. Not to mention that they make web development fun again.

To help you with your journey, we’ll start writing tutorials specifically for you, JavaScript developers, starting with a how-to-get-started guide and a cheat sheet.

Download & Update Node.js to the Latest Version! Node v21.5.0 Current / LTS v20.10.0 Direct Links

Node 20 is the active LTS version which will be supported until 22 Oct 2024, while Node 21 became the Current version in 2023 October: https://blog.risingstack.com/nodejs-21/

NAMERELEASEDACTIVE SUPPORTSECURITY SUPPORTLATEST
Node 21 CURRENT2023. Oct 17.2024. Apr 1.2024. Jun 1.21.5.0
Node 20 LTS2023. Apr 18.2024. Oct 22.2026. Apr 3020.10.0

In this article below, you’ll find changelogs and download / update information regarding Node.js!

Node.js LTS & Current Download for macOS:

Node.js LTS & Current Download for Windows:

For other downloads like Linux libraries, source codes, Docker images, etc. please visit https://nodejs.org/en/download/

Node.js Release Schedule:

Releases

Node.js v21 is the next Current version!

The latest major version of Node.js has just released with a few new interesting experimental features and a lot of fixes and optimization. You can find our highlights in this article: https://blog.risingstack.com/nodejs-21/

  • Built-in WebSocket client:
    A browser-compatible WebSocket implementation has been added to Node.js with this new release as an experimental feature. You can give it a go using the --experimental-websocket flag. The current implementation allows for opening and closing of websocket connections and sending data.

  • flush option for the writeFile type filesystem functions:
    Up until now, it was possible for data to not be flushed immediately to permanent storage when a write operation completed successfully, allowing read operations to get stale data. In response, a flush option has been added to the fs module file writing functions that, when enabled, forces data to be flushed at the end of a successful write operation using sync.

  • Addition of a global navigator Object:
    This new release also introduces a global navigator object to take steps towards enhancing web interoperability. We can now access hardware concurrency information through navigator.hardwareConcurrency, the only currently implemented method on the object.

  • Array grouping:
    There is a new static method added to Object and MapgroupBy(), that groups the items of a given iterable according to a provided callback function.

  • Additional changes:
    • Both the fetch and the webstreams modules are now marked as stable after receiving a few changes with this version.
    • A host of performance improvements as usual with any new release.
    • WebAssembly gets extended const expressions
    • Another new experimental flag, --experimental-default-type, has been added that allows setting the default module type to ESM
    • The globalPreload hook has been removed, it’s functionality replaced by register and initialize
    • Glob patterns are now supported in the test runner

Learn More Node.js from RisingStack

At RisingStack we’ve been writing JavaScript / Node tutorials for the community in the past 5 years. If you’re beginner to Node.js, we recommend checking out our Node Hero tutorial series! The goal of this series is to help you get started with Node.js and make sure you understand how to write an application using it.

See all chapters of the Node Hero tutorial series:
  1. Getting Started with Node.js
  2. Using NPM
  3. Understanding async programming
  4. Your first Node.js HTTP server
  5. Node.js database tutorial
  6. Node.js request module tutorial
  7. Node.js project structure tutorial
  8. Node.js authentication using Passport.js
  9. Node.js unit testing tutorial
  10. Debugging Node.js applications
  11. Node.js Security Tutorial
  12. How to Deploy Node.js Applications
  13. Monitoring Node.js Applications

As a sequel to Node Hero, we have completed another series called Node.js at Scale – which focuses on advanced Node / JavaScript topics. Take a look!


Node.js 21 is here with Websocket

The latest major version of Node.js has just released with a few new interesting experimental features and a lot of fixes and optimization. You can find our highlights from the release notes.

Built-in WebSocket client

A browser-compatible WebSocket implementation has been added to Node.js with this new release as an experimental feature. You can give it a go using the --experimental-websocket flag. The current implementation allows for opening and closing of websocket connections and sending data. There are four events available for use: open, close, message and error – so the basics are covered. It’s pretty exciting to see an out-of-the-box websocket implementation coming to Node, it could spare us the inclusion of yet another library in projects that need bidirectional communication. Be sure to give it a go and give your feedback to the developers!

A flush option for the writeFile type filesystem functions

Up until now, it was possible for data to not be flushed immediately to permanent storage when a write operation completed successfully, allowing read operations to get stale data. In response, a flush option has been added to the fs module file writing functions that, when enabled, forces data to be flushed at the end of a successful write operation using sync. This feature is not enabled by default, so make sure to include { flush: true } in the options if you’d like to use it.

Here is the list of functions the flush option has been added to:

  • filehandle.createWriteStream
  • fsPromises.writeFile
  • fs.createWriteStream
  • fs.writeFile
  • fs.writeFileSync

Addition of a global navigator Object

This new release also introduces a global navigator object to take steps towards enhancing web interoperability. We can now access hardware concurrency information through navigator.hardwareConcurrency, the only currently implemented method on the object. While this might not seem like a huge change for now, we can assume more and more functionality will be implemented with time, until we have the whole suit of information window.navigator provides in browser environments. This would spare us having to decide between using process and navigator in our code that is to be ran in both a browser and in Node.js.

Array grouping

There is a new static method added to Object and MapgroupBy(), that groups the items of a given iterable according to a provided callback function. The object returned contains a property for each group, whose value is an array with the items that belong to the group. In case of Object, the keys of the returned object will be strings, while the version on Map can have any kind of key.

Additional changes

  • Both the fetch and the webstreams modules are now marked as stable after receiving a few changes with this version.
  • A host of performance improvements as usual with any new release.
  • WebAssembly gets extended const expressions
  • Another new experimental flag, --experimental-default-type, has been added that allows setting the default module type to ESM
  • The globalPreload hook has been removed, it’s functionality replaced by register and initialize
  • Glob patterns are now supported in the test runner

Don’t forget that Node.js 16 is at its end of life, so if you’re still using this version, you should plan to upgrade soon to one of the newer LTS versions as soon as possible! The currently active LTS releases are 18 and 20, with version 22 – that is also an LTS version – scheduled to release in April 2024. You can find more information about the release schedule here.

The Best JavaScript Frameworks: Pros and Cons Explained

There are a lot of different JavaScript frameworks out there, and it can be tough to keep track of them all. In this article, we’ll focus on the most popular ones, and explore why they’re either loved or disliked by developers.

React

React is a JavaScript library for building user interfaces. It is maintained by Facebook and a community of individual developers and companies. React can be used as a base in the development of single-page or mobile applications. However, React is only concerned with rendering data to the DOM, and so creating React apps usually requires the use of additional libraries for state management, routing, and interaction with an API. React is also used for building reusable UI components. In that sense, it works much like a JavaScript framework such as Angular or Vue. However, React components are typically written in a declarative manner rather than using imperative code, making them easier to read and debug. Because of this, many developers prefer to use React for building UI components even if they are not using it as their entire front-end framework.

Advantages: 

  • React is fast and efficient because it uses a virtual DOM rather than manipulating the real DOM.
  • React is easy to learn because of its declarative syntax and clear documentation.
  • React components are reusable, making code maintenance easier.

Disadvantages: 

  • React has a large learning curve because it is a complex JavaScript library.
  • React is not a full-fledged framework, and so it requires the use of additional libraries for many tasks.

Next.js

Next.js is a javascript library that enables server-side rendering for React applications. This means that next.js can render your React application on the server before sending it to the client. This has several benefits. First, it allows you to pre-render components so that they are already available on the client when the user requests them. Second, it enables better SEO for your React application by allowing crawlers to index your content more easily. Finally, it can improve performance by reducing the amount of work that the client has to do in order to render the page.

Here’s why developers like Next.js: 

  • Next.js makes it easy to get started with server-side rendering without having to do any configuration.
  • Next.js automatically code splits your application so that each page is only loaded when it is requested, which can improve performance.

Disadvantages:

  • If you’re not careful, next.js can make your application codebase more complex and harder to maintain.
  • Some developers find the built-in features of next.js to be opinionated and inflexible.

Vue.js

Vue.js is an open-source JavaScript framework for building user interfaces and single-page applications. Unlike other frameworks such as React and Angular, Vue.js is designed to be lightweight and easy to use. The Vue.js library can be used in conjunction with other libraries and frameworks, or can be used as a standalone tool for creating front-end web applications. One of the key features of Vue.js is its two-way data binding, which automatically updates the view when the model changes, and vice versa. This makes it an ideal choice for building dynamic user interfaces. In addition, Vue.js comes with a number of built-in features such as a templating system, a reactivity system, and an event bus. These features make it possible to create sophisticated applications without having to rely on third-party libraries. As a result, Vue.js has become one of the most popular JavaScript frameworks in recent years.

Advantages: 

  • Vue.js is easy to learn due to its small size and clear documentation.
  • Vue.js components are reusable, which makes code maintenance easier.
  • Vue.js applications are very fast due to the virtual DOM and async component loading.

Disadvantages: 

  • While Vue.js is easy to learn, it has a large learning curve if you want to master all its features.
  • Vue.js does not have as many libraries and tools available as some of the other frameworks.

Angular

Angular is a JavaScript framework for building web applications and apps in JavaScript, html, and Typescript. Angular is created and maintained by Google. Angular provides two-way data binding, so that changes to the model are automatically propagated to the view. It also provides a declarative syntax that makes it easy to build dynamic UIs. Finally, Angular provides a number of useful built-in services, such as HTTP request handling, and support for routing and templates.

Advantages: 

  • Angular has a large community and many libraries and tools available.
  • Angular is easy to learn due to its well-organized documentation and clear syntax.

Disadvantages: 

  • While Angular is easy to learn, it has a large learning curve if you want to master all its features.
  • Angular is not as lightweight as some of the other frameworks.

Svelte

In a nutshell, Svelte is a JavaScript framework similar to React, Vue, or Angular. However, where those frameworks use virtual DOM (Document Object Model) diffing to figure out what changed between views, Svelte uses a technique called DOM diffing. This means that it only updates the parts of the DOM that have changed, making for a more efficient rendering process. In addition, Svelte also includes some built-in optimizations that other frameworks do not, such as automatically batching DOM updates and code-splitting. These features make Svelte a good choice for high-performance applications.

Advantages: 

  • Svelte has built-in optimizations that other frameworks do not, such as code-splitting.
  • Svelte is easy to learn due to its clear syntax and well-organized documentation.

Disadvantages: 

  • While Svelte is easy to learn, it has a large learning curve if you want to master all its features.
  • Svelte does not have as many libraries and tools available as some of the other frameworks.

Gatsby

Gatsby is a free and open-source framework based on React that helps developers build blazing fast websites and apps. It uses cutting edge technologies to make the process of building websites and applications more efficient. One of its key features is its ability to prefetch resources so that they are available instantaneously when needed. This makes Gatsby websites extremely fast and responsive. Another benefit of using Gatsby is that it allows developers to use GraphQL to query data from any source, making it easy to build complex data-driven applications. In addition, Gatsby comes with a number of plugins that make it even easier to use, including ones for SEO, analytics, and image optimization. All of these factors make Gatsby an extremely popular choice for building modern websites and applications.

Advantages: 

  • Gatsby websites are extremely fast and responsive due to its use of prefetching.
  • Gatsby makes it easy to build complex data-driven applications due to its support for GraphQL.
  • Gatsby comes with a number of plugins that make it even easier to use.

Disadvantages: 

  • While Gatsby is easy to use, it has a large learning curve if you want to master all its features.
  • Gatsby does not have as many libraries and tools available as some of the other frameworks.

Nuxt.js

Nuxt.js is a progressive framework for building JavaScript applications. It is based on Vue.js and comes with a set of tools and libraries that make it easy to create universal applications that can be rendered on server-side and client-side. Nuxt.js also provides a way to handle asynchronous data and routing, which makes it perfect for building highly interactive applications. In addition, Nuxt.js comes with a CLI tool that makes it easy to scaffold new projects and build, run, and test them. With Nuxt.js, you can create impressive JavaScript applications that are fast, reliable, and scalable.

Advantages: 

  • Nuxt.js is easy to use and extend.
  • Nuxt.js applications are fast and responsive due to server-side rendering.

Disadvantages: 

  • While Nuxt.js is easy to use, it has a large learning curve if you want to master all its features.
  • Nuxt.js does not have as many libraries and tools available as some of the other frameworks.

Ember.js

Ember.js is known for its conventions over configuration approach which makes it easier for developers to get started with the framework. It also features built-in libraries for common tasks such as data persistence and routing which makes development faster.  Although Ember.js has a steep learning curve, it provides developers with a lot of flexibility and power to create rich web applications. If you’re looking for a front-end JavaScript framework to build SPAs, Ember.js is definitely worth considering.

Advantages:  

  • Ember.js uses conventions over configuration which makes it easier to get started with the framework.
  • Ember.js has built-in libraries for common tasks such as data persistence and routing.
  • Ember.js provides developers with a lot of flexibility and power to create rich web applications.

Disadvantages: 

  • Ember.js has a steep learning curve.
  • Ember.js does not have as many libraries and tools available as some of the other frameworks.

Backbone.js

Backbone.js is a lightweight JavaScript library that allows developers to create single-page applications. It is based on the Model-View-Controller (MVC) architecture, which means that it separates data and logic from the user interface. This makes code more maintainable and scalable, as well as making it easier to create complex applications. Backbone.js also includes a number of features that make it ideal for developing mobile applications, such as its ability to bind data to HTML elements and its support for touch events. As a result, Backbone.js is a popular choice for developers who want to create fast and responsive applications.

Advantages:  

  • Backbone.js is lightweight and only a library, not a complete framework.
  • Backbone.js is easy to learn and use.
  • Backbone.js is very extensible with many third-party libraries available.

Disadvantages: 

  • Backbone.js does not offer as much built-in functionality as some of the other frameworks.
  • Backbone.js has a smaller community than some of the other frameworks.

Conclusion

In conclusion, while there are many different JavaScript frameworks to choose from, the most popular ones remain relatively stable. Each has its own benefits and drawbacks that developers must weigh when making a decision about which one to use for their project. While no framework is perfect, each has something to offer that can make development easier or faster. 

Everyone should consider the specific needs of their project when choosing a framework, as well as the skills of their team and the amount of time they have to devote to learning a new framework. By taking all of these factors into account, you can choose the best JavaScript framework for your project!

ChatGPT use case examples for programming

If you’re reading this post, you probably already know enough about large language models and other “AI” tools, so we can skip the intro

Despite the fact that the “AI is going to take our jobs” discourse proved to be an effective tool in the clickbait content creators toolbelt, I will not take this road.

Instead of contributing to the moral panic about the supposedly inevitable replacement of white collar jobs, or pretending to be offended by a chatbot, I’ll help our readers to consider GPT-based products as tools that could be useful in a professional webdev setting. 

To do so, I asked some of my colleagues about their experiences of using GPT and various mutations of it – to help you get a more grounded understanding of their utility.

In case you have an experience that you consider useful sharing with the RisingStack community, please share it though this form. 

I’ll drop the results / best ones in the article later on!

Daniel’s  ‘Code GPT’ vscode plugin review

I’ve been pretty satisfied with GitHub Copilot. It does the job well, and it is priced reasonably. Still, after depleting the free tier, I decided to look for an open source alternative.

TabNine is an honorable mention here, and a well established player, but based on my previous experience (about two years ago, mind you), it is clunky. Nowhere near the breeze of a dev experience you get from Copilot.

But take heart, there is a staggering amount of plugins out there for VS Code, if you look for AI-based coding assistants. 

At the time of writing this, Code GPT is the winner by number of downloads, and number of (positive) votes, so I decided to give it a go. You can choose from a range of OpenAPI and Cohere models, with GPT-3 being the default.

Features:

1, Code Generation from comment prompts

The suggestions are relevant, and of quality. The plugin doesn’t offer code completion on the fly, unlike Copilot, but communicates with you in a new IDE pane it opens automatically instead. I like this feature, since I can pick the parts from the suggestion I liked, without bloating the code I’m working on, and having to delete the irrelevant lines. This behavior comes in handy with the other features as well. Let’s see those.

2, Unit Test generation

While the results are often far from being complete, it saves me a lot of boilerplate code. It is also handy in reminding me of cases that I otherwise might have forgotten. For this feature to work well, adjust your max token length to a 1000 at least in the Settings, since a comprehensive test suite usually ends up quite verbose, and you’ll only get part of it with a tight quota.

3, Find Problems

Your code review buddy. Once I feel I’m done with my work, a quick scan doesn’t take long before committing. While it often is straight out wrong about the ‘issues’ it points out, it doesn’t take long to scan through the suggestions, and catch mistakes before your real life reviewer does.

4, Refactor

Save some time for your team lead for extra credits, and run Refactor against your code. Don’t expect miracles to happen, but often times it catches stuff that managed to sneak under your radar. Note: the default max token length won’t cut it here either.

5, Document and Explain

Listed as two separate functionality in the documentation, it achieves essentially the same thing; provides a high level natural language description on what the highlighted peace of code does. I tend to use it less often, but it is a nice to have.

6, Ask CodeGPT

I left it the last, but this is the most flexible feature of this plugin. It can achieve all previously mentioned functionalities with the right prompt, and more. Convert your .js to .ts,  generate a README.md file from code, as suggested in the documentation, or just go ahead and ask for a recipe for a delicious apple pie, like you would from ChatGPT 🥧 

My Conclusion:

Code GPT offers many functionality that Copilot doesn’t, but lacks the thing Copilot is best at: inline code completion. So if you want to take the most out of AI, just use both, as these two tools complement each other really nice.

Code GPT Might come handy if you’re just getting started with a new language or framework. The Explain feature helps double-check your gut feeling, or gives you the missing hint in the right direction.

Bump up your max token length to at least a 1000, c’mon, it’s only ¢2 😉

An interesting alternative I might be trying in the future is ‘ChatGPT’ plugin (from either Tim Kmecl or Ali Gencay) that claims to be using the unofficial Chat GPT API, with all its superpowers.

Further reading:

– official site: https://www.codegpt.co/

– GitHub CoPilot vs ChatGPT: https://dev.to/ruppysuppy/battle-of-the-giants-github-copilot-vs-chatgpt-4oac

– List of GitHub CoPilot alternatives: https://www.tabnine.com/blog/github-copilot-alternatives/

Olga on writing Mongo queries with ChatGPT

I have used ChatGPT for more effective coding. It was really helpful for example with enhancing Mongo queries for more complex use cases as it suggested specific stages that worked for a use case, which would have definitely taken me more time to research and realize which stage and/or operator is ideal for this query. 

However all the answers it produces should be checked and not used blindly. I have not yet come across a case when the answer it provided didn’t need modification (though maybe it is due to the fact that I didn’t use it for easy things). 

I have also noticed that, if a question posted to ChatGPT includes many different parameters, in a lot of cases it will not take them all to consideration so one has to continue conversation and ensure all parameters are considered in the solution.

Akos on using ChatGPT instead of StackOverflow

I have been using ChatGPT since its inception and have found it to be a valuable tool in my daily work. With ChatGPT, I no longer have to spend hours searching and Googling for regex patterns on Stack Overflow. Instead, I simply write down what I want with the regex, and the tool returns the result, saving me a significant amount of time and effort.

In addition to regex, I have also found ChatGPT to be a valuable tool when working on scrapers. Dealing with deeply nested selectors can be a challenge, and understanding how they work with scraping tools can take hours of research. But with ChatGPT, I can simply paste an example HTML and ask the tool to select what I want, saving me even more time and effort.

However, it is important to use ChatGPT in moderation. Overusing the tool could lead to a decline in my problem-solving skills and make me too dependent on it. By setting limits, I can still benefit from ChatGPT’s time-saving features while maintaining my critical thinking skills.

In conclusion, ChatGPT has been a powerful tool in my work, saving me time and effort in regex and scraping tasks. However, it is important to use it in moderation and not let it compromise my problem-solving skills.

Me, Navigating the hallucinations of chatGPT

I work with a lot of text: business documents, project proposals, contracts, articles, marketing materials, you name it. I’ve been fascinated by the emergence of “AI” tools in 2022 and both the entertainment factor and the usefulness they can provide.

All big areas – text, voice, image – had huge releases that made the internet a very interesting place again after the boring years of the web3 / crypto / nft / metaverse nonsense. Suddenly I felt the rush of discovering the internet in the early 2000’s. “Wow, I can do that?!” was a sentence that got repeated a lot this year.

The Dall-E2 / Stable Diffusion / Whisper releases were more than enough to shake up the web, but ChatGPT was the one that really disrupted the web. The first good conversational “AI” that everyone could use, and for free (for now). At times it can be so convincing that people forget that it is just a very good bullshitter. For example when I ask it about RisingStack, I get the following answer:

“RisingStack is a software development and consulting company that specializes in building scalable, high-performance systems using Node.js, Kubernetes, and other modern web technologies. The company was founded in 2013 and is headquartered in Budapest, Hungary, with additional offices in London, United Kingdom, and New York, United States. RisingStack offers a range of services, including software development, architecture design, cloud infrastructure management, performance optimization, and training. They also develop open-source tools such as Trace, a Node.js performance monitoring and debugging tool, and Pino, a Node.js logging library. RisingStack has worked with a variety of clients, ranging from small startups to large enterprises, across a range of industries, including fintech, healthcare, e-commerce, and media. They are known for their expertise in Node.js and their commitment to delivering high-quality software solutions.”

Most of this text is usable, but it contains a lot of outright lies. It is important to keep in mind that GPT hallucinates. It has no idea about what is true and what is not. It pretends to know things, but it’s just making things up word by word. In this case: RisingStack was founded in 2014, and we never had a London office. Trace was sunset like 6 years ago, and Pino has nothing to do with us.

Anyways, I find it really useful when I need to generate / rephrase / improve text. It is only valid as a Google replacement if you can confidently tell if it’s right or wrong, so “geepeetee-ing” something is not really that helpful right now.

I already used it to write contracts, blog posts (not this one though), business proposals. It also brought in new clients, as just in the past couple of weeks we used it to..

  • Automatically generate product names and descriptions for webshops
  • Create easy-read text for children with disabilities
  • Perform sentiment analysis and write answers automatically to customer reviews

Currently chatGPT has a lame writing style by default. It’s very formulaic. I’ve seen so much of it that I believe I can spot it 8 out of 10 times right away. It lies a lot, and I wasn’t able to get anything guitar-related useful out of it, despite the fact that the training material probably has a couple million tabs in it. 

Anyways, here are my not-so-hot takes to about it:

  • You really need to carefully double check everything you generate. On the surface most of it might look good enough, but that’s just making it easier for everyone to get lazy with it.
  • “AI” won’t replace jobs, instead, it will just improve productivity. As Photoshop is a better brush, GPT should be thought of as a better text/code editor. Most of the office jobs are about collaboration anyways, not typing on a keyboard.
  • Artists won’t get replaced en masse. You won’t be able to prompt an engine to generate artwork in de Goya’s style, if cave paintings are the apex of your visual art knowledge. Taste will be very important to stand out when the web gets flooded with endless mediocre “art”. Also..
  • It will be interesting to see how the “poisoning the well” problem will affect these models. The continuous retraining of the “AI” on already “AI generated” content will cause a big decline in the quality of these services, in case they won’t be able to filter them out… While they are working on making the generated content so good that it gets mistaken for genuine human creation.
  • It’s a bit scary to think about how Microsoft will dominate this space through its OpenAI investment. Despite the genius branding, it is not open at all, and will cost a lot of money without serious competitors or general access to free-to-use alternatives (like Stable Diffusion for images).
  • Most of the coverage GPT gets nowadays is about people gaming the engine to finally say something “bad”, then pretending to be offended, even more so, scared of it! This kind of AI ethics/alignment discourse is incredibly dull and boring, imho..
  • Although the adversarial aspect is very interesting. Poisoning generally available chatbots training data will be a prime trolling activity, while convincing chatbots to spill their carefully crafted secret sauce prompts is something that needs to be continuously prevented.

I was first skeptical about prompt engineering as an emerging “profession”, but seeing how building products on top of GPT3 requires proper prompting and safeguards to make the end result consistently useful for end users, I can see it happening. Also, when you build something LLM driven, you need to be aware that hostile users, trolls, competitors, etc.. will try to game your product to ramp up your cloud costs or cause reputational harm.

This tweet really gets it.

This is already happening. Most of the “AI-driven” products are just purpose-repacked custom prompters calling GPT3 through an API, with a fancy UI.

Anyways, I’m looking forward to seeing what kind of GPT driven products we’ll make for our clients, and how the internet will change in general.

I’m curious about your experience with chatGPT, so please share it with me through this short form!

Cheers,

Ferenc

AI Development Tools Compared – The Differences You’ll Need to Know

There are many different types of AI development tools available, but not all of them are created equal. Some tools are more suited for certain tasks than others, and it’s important to select the right tool for the job.

Choosing the wrong tool can lead to frustration and wasted time, so it’s important to do your research before you start coding. There are many different types of AI development tools available, so there’s sure to be one that fits your needs. Common types of AI development tools include cloud-based platforms, open source software, and low code development tools. 

Cloud-based platforms are typically the most user friendly and allow you to build sophisticated models quickly. They offer a wide variety of features, such as data analysis tools, natural language processing capabilities, automatic machine learning models creation and pre-trained models that can be used for various tasks.

Open-source software offers a great deal of flexibility and the ability to customize your AI model for specific tasks. However, using open source software requires coding knowledge and experience and is best suited for more experienced developers. 

Low code development tools allow you to create AI applications without having to write code. These tools allow developers of any skill level to quickly and easily create AI applications, eliminating the need for coding knowledge or experience. 

Of course, there are occasional overlaps, like cloud platforms using open-source technologies – but to find out all the similarities and differences, we’ll need to examine them. Let’s explore each one in further detail:

Detectron 2

Detectron 2 is Facebook’s state-of-the-art object detection and segmentation library. It features a number of pre-trained models and baselines that can be used for a variety of tasks, and it also has cuda bindings that allow it to run on gpu for even faster training. Compared to its predecessor, Detectron 2 is much faster to train and can achieve better performance on a variety of benchmarks. It is also open source and written in python, making it easy to use and extend. Overall, Detectron 2 is an excellent choice for any object detection or segmentation task.

The fact that it is built on PyTorch makes it very easy to share models between different use cases. For example, a model that is developed for research purposes can be quickly transferred to a production environment. This makes Detectron2 ideal for organizations that need to move quickly and efficiently between different use cases. In addition, the library’s ability to handle large-scale datasets makes it perfect for organizations that need to process large amounts of data. Overall, Detectron2 is an extremely versatile tool that can be used in a variety of different settings.

Caffe

Caffe is a deep learning framework for model building and optimisation. It was originally focused on vision applications, but it is now branching out into other areas such as sequences, reinforcement learning, speech, and text. Caffe is written in C++ and CUDA, with interfaces for python and mathlab. The community has built a number of models which are available at https://github.com/BVLC/caffe/wiki/Model-Zoo. Caffe is a powerful tool for anyone interested in deep learning.

It features fast, well-tested code and a seamless switch between CPU and GPU – meaning that if you don’t have a GPU that supports CUDA, it automatically defaults to the CPU. This makes it a versatile tool for deep learning researchers and practitioners. The Caffe framework is also open source, so anyone can contribute to its development.

Caffe offers the model definitions, optimization settings, and pre-trained weights so you can start right away. The BVLC models are licensed for unrestricted use, so you can use them in your own projects without any restrictions.

Keras

Keras is a deep learning framework that enables fast experimentation. It is based on Python and supports multiple backends, including TensorFlow, CNTK, and Theano. Keras includes specific tools for computer vision (KerasCV) and natural language processing (KerasNLP). Keras is open source and released under the MIT license.

The idea behind Keras is to provide a consistent interface to a range of different neural network architectures, allowing for easy and rapid prototyping. It is also possible to run Keras models on top of other lower-level frameworks such as MXNet, Deeplearning4j, TensorFlow or Theano. Keras, like other similar tools, has the advantage of being able to run on both CPU and GPU devices with very little modification to the code.

In addition, Keras includes a number of key features such as support for weight sharing and layer reuse, which can help to improve model performance and reduce training time.

CUDA

The CUDA toolkit is a powerful set of tools from NVIDIA for running code on GPUs. It includes compilers, libraries, and other necessary components for developing GPU-accelerated applications. The toolkit supports programming in Python, C, and C++, and it makes it easy to take advantage of the massive parallel computing power of GPUs. With the CUDA toolkit, you can accelerate your code to run orders of magnitude faster than on a CPU alone. Whether you’re looking to speed up machine learning algorithms or render complex 3D graphics, the CUDA toolkit can help you get the most out of your NVIDIA GPUs.

In the context of fraud detection, the CUDA toolkit can be used to train graph neural networks (GNNs) on large datasets in an efficient manner. This allows GNNs to learn from more data, which can lead to improved performance. In addition, the CUDA toolkit can be used to optimize the inference process, which is important for real-time applications such as fraud detection, which is a critical application for machine learning. Many techniques struggle with fraud detection because they cannot easily identify patterns that span multiple transactions. However, GNNs are well-suited to this task due to their ability to aggregate information from the local neighborhood of a transaction. This enables them to identify larger patterns that may be missed by traditional methods.

TensorFlow

TensorFlow is an open-source platform for machine learning that offers a full pipeline from model building to deployment. It has a large collection of pre-trained models and supports a broad range of programming languages including Javascript, Python, Android, Swift, C++, and Objective C. TensorFlow uses the Keras API and also supports CUDA for accelerated training on NVIDIA GPUs. In addition to providing tools for developers to build and train their own models, TensorFlow also offers a wide range of resources such as tutorials and guides.

TensorFlow.js is a powerful tool that can be used to solve a variety of problems. In the consumer packaged goods (CPG) industry, one of the most common problems is real-time and offline SKU detection. This problem is often caused by errors in manually inputting data, such as when a product is scanned at a store or when an order is placed online. TensorFlow.js can be used to create a solution that would automatically detect and correct these errors in real time, as well as provide offline support for cases where a connection is not available. This can greatly improve the efficiency of the CPG industry and reduce the amount of waste caused by incorrect data input.

PyTorch

PyTorch is a powerful machine learning framework that allows developers to create sophisticated applications for computer vision, audio processing, and time series analysis. The framework is based on the popular Python programming language, and comes with a large number of libraries and frameworks for easily creating complex models and algorithms. PyTorch also supports bindings for c++ and java, making it a great option for cross-platform development. In addition, the framework includes CUDA support for accelerated computing on NVIDIA GPUs. And finally, PyTorch comes with a huge collection of pre-trained models that can be used for quickly building sophisticated applications.

PyTorch’s ease of use and flexibility make it a popular choice for researchers and developers alike. The PyTorch framework is known to be convenient and flexible, with examples covering reinforcement learning, image classification, and natural language processing as the more common use cases. As a result, it is no surprise that the framework has been gaining popularity in recent years. Thanks to its many features and benefits, PyTorch looks poised to become the go-to framework for deep learning in the years to come.

Apache MXNet

MXNet is an open-source deep learning framework that allows you to define, train, and deploy deep neural networks on a wide array of devices, from cloud infrastructure to mobile devices. It’s scalable, allowing for fast model training, and supports a flexible programming model and multiple languages.

It’s built on a dynamic dependency scheduler that automatically parallelizes both symbolic and imperative operations on the fly. A graph optimization layer makes symbolic execution fast and memory efficient.

The MXNet library is portable and lightweight. It’s accelerated with the NVIDIA Pascal™ GPUs and scales across multiple GPUs and multiple nodes, allowing you to train models faster. Whether you’re looking to build state-of-the-art models for image classification, object detection, or machine translation, MXNet is the tool for you.

Horovod

Horovod is a distributed training framework for deep learning that supports TensorFlow, Keras, PyTorch, and Apache MXNet. It is designed to make distributed training easy to use and efficient. Horovod uses a message passing interface to communicate between nodes, and each node runs a copy of the training script. The framework handles the details of communication and synchronization between nodes so that users can focus on their model. Horovod also includes a number of optimizations to improve performance, such as automatically fusing small tensors together and using hierarchical allreduce to reduce network traffic.

For Uber’s data scientists, the process of installing TensorFlow was made even more challenging by the fact that different teams were using different releases of the software. The team wanted to find a way to make it easier for all teams to use the ring-allreduce algorithm, without requiring them to upgrade to the latest version of TensorFlow or apply patches to their existing versions. The solution was to create a stand-alone package called Horovod. This package allowed the team to cut the time required to install TensorFlow from about an hour to a few minutes, depending on the hardware. As a result, Horovod has made it possible for Uber’s data scientists to spend less time installing software and more time doing what they do best.

Oracle AI

Oracle AI is a suite of artificial intelligence services that can be used to build, train and deploy models. The services include natural language processing, chat bots / customer support, text-to-speech, speech-to-text, object detection for images and data mining. Oracle AI offers pre-configured vms with access to GPUs. The service can be used to build models for anomaly detection, analytics and data mining. Oracle AI is a powerful tool that can be used to improve your business.

Children’s Medical Research Institute (CMRI) is a not-for-profit organisation dedicated to improving the health of children through medical research. CMRI moved to Oracle Cloud Infrastructure (OCI) as its preferred cloud platform. This move has helped the institute take advantage of big data and machine learning capabilities to automate routine database tasks, database consolidation, operational reporting, and batch data processing. Overall, the switch to OCI has been a positive move for CMRI, and one that is sure to help the institute continue its important work.

H2O

H2O is a powerful open source AI platform that is used by companies all over the world to improve their customer support, marketing, and data mining efforts. The software provides a wide range of features that make it easy to collect and analyze customer data, identify anomalies, and create chat bots that can provide an engaging customer experience. H2O is constantly evolving, and the company behind it is always introducing new features and improvements.

For example, it can be used to create an intelligent cash management system that predicts cash demand and helps to optimize ATM operations. It can also help information security teams reduce risk by identifying potential threats and vulnerabilities in real time. In addition, H2O.AI can be used to transform auditing from quarterly to real-time, driving audit quality, accuracy and reliability.

Alibaba Cloud

Alibaba Cloud is a leading provider of cloud computing services. Its products include machine learning, natural language processing, data mining, and analytics. Alibaba Cloud’s machine learning platform offers a variety of pre-created algorithms that can be used for tasks such as data mining, anomaly detection, and predictive maintenance. The platform also provides tools for training and deploying machine learning models. Alibaba Cloud’s natural language processing products offer APIs for text analysis, voice recognition, and machine translation. The company’s data mining and analytics products provide tools for exploring and analyzing data. Alibaba Cloud also offers products for security, storage, and networking.

Alibaba, the world’s largest online and mobile commerce company, uses intelligent recommendation algorithms to drive sales using personalized customer search suggestions on its Tmall homepage and mobile app. The system takes into account a customer’s purchase history, browsing behavior, and social interactions when making recommendations. Alibaba has found that this approach leads to increased sales and higher customer satisfaction. In addition to search suggestions, the system also provides personalized product recommendations to customers based on their past behavior. This has resulted in increased sales and engagement on the platform. Alibaba is constantly tweaking and improving its algorithms to ensure that it is providing the most relevant and useful data to its users.

IBM Watson

IBM Watson is a powerful artificial intelligence system that has a range of applications in business and industry. One of the most important functions of Watson is its ability to process natural language. This enables it to understand human conversation and respond in a way that sounds natural. This capability has been used to develop chatbots and customer support systems that can replicate human conversation. In addition, Watson’s natural language processing capabilities have been used to create marketing campaigns that can target specific demographics. Another key application of Watson is its ability to detect anomalies. This makes it an essential tool for monitoring systems and identifying potential problems. As a result, IBM Watson is a versatile and valuable artificial intelligence system with a wide range of applications.

IBM Watson is employed in nearly every industry vertical, as well as in specialized application areas such as cybersecurity. This technology is often used by a company’s data analytics team, but Watson has become so user friendly that it is also easily used by end users such as physicians or marketers.

Azure AI

Azure AI is a suite of services from Microsoft that helps you build, optimize, train, and deploy models. You can use it for object detection in images and video, natural language processing, chatbots and customer support, text-to-speech, speech-to-text, data mining and analytics, and anomaly detection. Azure AI also provides pre-configured virtual machines so you can get started quickly and easily. Whether you’re an experienced data scientist or just getting started with machine learning, Azure AI can help you achieve your goals.

With the rapid pace of technological advancement, it is no surprise that the aviation industry is constantly evolving. One of the leading companies at the forefront of this change is Airbus. The company has unveiled two new innovations that utilize Azure AI solutions to revolutionize pilot training and predict aircraft maintenance issues.

Google AI

Google AI is a broad set of tools and services that helps you build, deploy, and train models, as well as to take advantage of pre-trained models. You can use it to detect objects in images and video, to perform natural language processing tasks such as chat bots or customer support, to translate text, and to convert text-to-speech or speech-to-text. Additionally, Google AI can be used for data mining and analytics, as well as for anomaly detection. All of these services are hosted on Google Cloud Platform, which offers a variety of options for GPU-accelerated computing, pre-configured virtual machines, and TensorFlow hosting.

UPS and Google Cloud Platform were able to develop routing software that has had a major impact on the company’s bottom line. The software takes into account traffic patterns, weather conditions, and the location of UPS facilities, in order to calculate the most efficient route for each driver. As a result, UPS has saved up to $400 million a year, and reduced its fuel consumption by 10 million gallons. In addition, the software has helped to improve customer satisfaction by ensuring that packages are delivered on time.

AWS AI

Amazon Web Services offers a variety of AI services to help developers create intelligent applications. With pre-trained models for common use cases, AWS AI makes it easy to get started with machine learning. For images and video, the object detection service provides accurate labels and coordinates. Natural language processing can be used for chat bots and customer support, as well as translation. Text-to-speech and speech-to-text are also available. AI powered search provides relevant results from your data. Pattern recognition can be used for code review and monitoring. And data mining and analytics can be used for anomaly detection. AWS AI also offers hosted GPUs and pre-configured vms. With so many powerful features, Amazon Web Services is the perfect platform for developing AI applications.

Formula 1 is the world’s most popular motorsport, with hundreds of millions of fans worldwide. The sport has been at the forefront of technological innovation for decades, and its use of data and analytics has been central to its success. Teams have long used on-premises data centers to store and process large amounts of data, but the sport is now accelerating its transformation to the cloud. Formula 1 is moving the vast majority of its infrastructure to Amazon Web Services (AWS), and standardizing on AWS’s machine-learning and data-analytics services. This will enable Formula 1 to enhance its race strategies, data tracking systems, and digital broadcasts through a wide variety of AWS services—including Amazon SageMaker, AWS Lambda, and AWS’s event-driven serverless computing service. By using these services, Formula 1 will be able to deliver new race metrics that will change the way fans and teams experience racing.

Conclusion

Choosing the right AI development tool can be difficult. This article has provided a comparison of some of the most popular tools on the market. Each tool has its own strengths and weaknesses, so it is important to decide which one will best suit your needs.

Kubernetes Interview Questions and Answers You’ll Need the Most

Are you currently preparing for a Kubernetes interview? If so, you’ll want to make sure you’re familiar with the questions and answers below at least. This article will help you demonstrate your understanding of Kubernetes concepts and how they can be applied in practice. With enough preparation, you’ll be able to confidently nail your next interview and showcase your Kubernetes skills. Let’s get started!

What is Kubernetes?

Kubernetes is a platform for managing containerized stateless or stateful applications across a cluster of nodes. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. Kubernetes also automates the replication of the containers across multiple nodes in a cluster, as well as healing of failed containers. Kubernetes was originally designed by Google, and is now maintained by the Cloud Native Computing Foundation.

Some of the key features of Kubernetes include:

– Provisioning and managing containers across multiple hosts

– Scheduling and deploying containers

– Orchestrating containers as part of a larger application

– Automated rollouts and rollbacks

– Handling container health and failure

– Scaling containers up and down as needed

– It has a large and active community that develops new features and supports users.

– It has a variety of tools for managing storage and networking for containers. 

What are the main differences between Docker Swarm and Kubernetes?

Docker Swarm and Kubernetes are both container orchestration platforms. They are both designed for deploying and managing containers at scale. However, there are some key differences between the two platforms.

Docker Swarm is a native clustering solution for Docker. It is simpler to install and configure than Kubernetes. Docker Swarm also uses the same CLI and API as Docker, so it is easy to learn for users who are already familiar with Docker. However, Docker Swarm lacks some of the advanced features that Kubernetes has, such as automatic rollouts and rollbacks, health checks, and secrets management.

Kubernetes is a more complex system than Docker Swarm, but it offers a richer feature set. Kubernetes is also portable across different environments, so it can be used in on-premise deployments, as well as cloud-based deployments. In addition, Kubernetes is backed by a large community of users and developers, so there is a wealth of support and documentation available.

To sum up:

-Kubernetes is more complicated to set up but the benefits are a robust cluster and auto-scaling 

-Docker Swarm is easy to set up but does not have a robust cluster or autoscaling 

What is a headless service?

​​A headless service is a special type of Kubernetes service that does not expose a cluster IP address. This means that the service will not provide load balancing to the associated pods. Headless services are useful for applications that require a unique IP per instance or for applications that do not require load balancing. For example, stateful applications such as databases often require a unique IP address per instance. By using a headless service, each instance can be given its own IP address without the need for a load balancer. Headless services can also be used to expose individual instances of an application outside of the Kubernetes cluster. This is often done by using a tool like kubectl to expose individual pods.

What are the main components of Kubernetes architecture?

Pods and containers are two components of a Kubernetes architecture. Pods are composed of one or more containers that share an IP address and port space. This means that containers within a pod can communicate with each other without going through a network. Pods also provide a way to deploy applications on a cluster in a replicable and scalable way. Containers, on the other hand, are isolated from each other and do not share an IP address. This isolation provides a higher level of security as each container can only be accessed by its own process. In addition, containers have their own file system, which means that they can be used to package up an application so that it can be run in different environments.

What are the different management and orchestrator features in Kubernetes?

The available management and orchestrator features in Kubernetes are: 

1. Cluster management components: These components manage the Kubernetes cluster.

2. Container orchestration components: These components orchestrate the deployment and operation of containers.

3. Scheduling components: These components schedule and manage the deployment of containers on nodes in the cluster.

4. Networking components: These components provide networking capabilities for containers in the cluster.

5. Storage components: These components provide storage for containers in the cluster.

6. Security components: These components provide security for the containers in the cluster.

What is the load balancer in Kubernetes?

A load balancer is a software program that evenly distributes network traffic across a group of servers. It is used to improve the performance and availability of applications that run on multiple servers.

Specifically, the load balancer in Kubernetes is a component that distributes traffic across nodes in a Kubernetes cluster. It can be used to provide high availability and to optimize resource utilization. Also, the load balancer can help to prevent overloads on individual nodes. 

What is Container resource monitoring?

Container resource monitoring means that you can keep track of CPU, Memory, and Disk space utilization for each container in your Kubernetes cluster. There are a two main ways to monitor the Kubernetes cluster. One way is to use the built-in kubectl command-line interface: this is able to monitor CPU utilization, memory usage and disk space. If you need to keep track of more data, then there’s another way: to use a third-party monitoring tool such as Datadog, New Relic, or Prometheus. 

What is the difference between a ReplicaSet and replication controller?

In Kubernetes, a ReplicaSet is a collection of pods that are always up and running. The replication controller’s objectives are to ensure that a desired number of pod replicas are running at all times, and to maintain the desired state of the pods in the system.

A ReplicaSet is a newer, more advanced concept that replaces replication controllers. A ReplicaSet allows you to define a minimum number of pods that must be up and running at all times, and provides a richer set of features than replication controllers.

ReplicaSets are the basic building blocks of Kubernetes clusters. They provide the ability to have multiple copies of an application running in parallel, and to scale out (add more nodes) or scale in (remove nodes) the number of copies as needed. Replication controllers provide the ability to maintain a desired number of pod replicas for a particular application.

A ReplicaSet ensures that a specified number of pod replicas are running at any given time. However, a Deployment is a higher-level concept that manages ReplicaSets and provides declarative updates to Pods along with a lot of other useful features. Therefore, we recommend using Deployments instead of directly using ReplicaSets, unless you require custom update orchestration or don’t require updates at all.

What are the recommended security measures for Kubernetes?

There are a number of recommended security measures for Kubernetes, including implementing third-party authentication and authorization tools, using network segmentation to restrict access to sensitive data, and maintaining regular monitoring and auditing of the cluster.

Another key recommendation is to use role-based access control (RBAC) to limit access to the Kubernetes API. This ensures that only authorized users can make changes to the system and introduces an additional layer of protection against potential vulnerabilities or attacks.

Node isolation is also worth mentioning. It is a process of isolating individual nodes in a Kubernetes cluster so that each node only has access to its own resources. This process is used to improve the security and performance of Kubernetes clusters by preventing malicious activity on one node from affecting other nodes. Node isolation can be achieved through a variety of means, such as using a firewall to block network traffic between nodes, or using software-defined networking to segment node traffic. By isolating nodes, Kubernetes administrators can ensure that each node in a cluster is used only for its intended purpose and that unauthorized access to resources is prevented.

Other best practices for securing Kubernetes include: 

– Restricting access to the Kubernetes API to authorized users only

– Using network firewalls to restrict access to the Kubernetes nodes from unauthorized users

– Using intrusion detection/prevention systems to detect and prevent unauthorized access to the Kubernetes nodes

– Using encryption for communications between the nodes and pods in the cluster

– Limiting which IP addresses have access to cluster resources

– Implementing regular vulnerability assessments.

Ultimately, incorporating these types of security measures into your Kubernetes deployment will help ensure the safety and integrity of your system.

What is Container Orchestration and how does it work in Kubernetes?

Container orchestration is the process of managing a group of containers as a single entity. Container orchestration systems, like Kubernetes, allow you to deploy and manage containers across a cluster of nodes. This provides a higher-level of abstraction and makes it easier to manage and scale your applications.

Kubernetes supports features for container orchestration, including:

– Creating and managing containers

– Configuring and managing networking

– Configuring and managing storage

– Booting and managing VMs

– Deploying applications

– Managing workloads

– Accessing logs and monitoring resources

– Configuring security and authentication

What are the features of Kubernetes?

Kubernetes is a platform that enables users to deploy, manage and scale containerized applications. Some of its key features include:

-Declarative syntax: Kubernetes uses a declarative syntax that makes it easy to describe the desired state of an application.

-Self-healing: Kubernetes is able to automatically heal applications and nodes in the event of failures.

-Horizontal scalability: Kubernetes enables users to scale their applications horizontally, by adding or removing nodes as needed.

-Fault tolerance: Kubernetes is able to tolerate failures of individual nodes or pods, ensuring that applications are always available.

What is Kube-apiserver and what’s the role of it?

The Kubernetes apiserver is a critical part of a Kubernetes deployment.

The apiserver provides a REST API for managing Kubernetes resources.

It also provides authentication and authorization for accessing those resources.

The apiserver must be secured to prevent unauthorized access to Kubernetes resources.

Use role-based access control to restrict access to specific resources.

What is a node in Kubernetes?

A node is a master or worker machine in Kubernetes. It can be a physical machine or a virtual machine.

A node is a member of a Kubernetes cluster. Each node in a Kubernetes cluster is assigned a unique ID, which is used to identify the node when communicating with the Kubernetes API.

When a new node is added to a Kubernetes cluster, the Kubernetes API is contacted to register the node with the cluster. The Kubernetes API stores information about the node, including its assigned ID, the addresses of the node’s Kubernetes masters, and the labels assigned to the node.

When a node is removed from a Kubernetes cluster, the Kubernetes API is contacted to unregister the node from the cluster. The Kubernetes API removes information about the node from its database, including the node’s assigned ID, the addresses of the node’s Kubernetes masters, and the labels assigned to the node.

What is kube-scheduler and what’s the role of it?

Kube-scheduler is responsible for keeping track of the state of the cluster and ensuring that all desired pods are scheduled.

In a Kubernetes cluster, the scheduler is responsible for assigning Pods to Nodes. 

When a new Pod is created, the scheduler watches for it and becomes responsible for finding the best Node for that Pod to run on. To do this, the scheduler looks at the requirements of the Pod and compares them with the capabilities of the Nodes in the cluster. The scheduler also takes into account factors such as Node utilization and available resources. By finding the best match between Pods and Nodes, the scheduler helps to ensure that Pods are running on an optimal Node. This, in turn, helps to improve the performance of the overall cluster.

To get the most out of the Kubernetes scheduler, you should configure it to schedule your pods as efficiently as possible. You can do this by configuring the scheduler’s resource constraints and pod priorities.

What is Minikube?

Minikube is important because it allows you to have a local Kubernetes environment. Minikube is a single node Kubernetes environment that you can install on your laptop. This is important because it allows you to develop and test Kubernetes applications without having to deploy them to a cluster.

What is a Namespace in Kubernetes?

Namespaces are a way to logically group objects in Kubernetes. By default, Kubernetes has a single namespace. Objects in different namespaces can have different security contexts and can be managed independently.

How can you handle incoming data from external sources (ingress traffic)?

Ingress is a Kubernetes resource that allows an organization to control how external traffic is routed to and from its services. Ingress resources are defined in a YAML file. An Ingress controller is then deployed to manage the ingress resource.

Ingress controllers use the Ingress Resource Definition to determine how to route traffic to services.

Ingress controllers can use a variety of methods to route traffic, including:

-Using a load balancer

-Using a DNS server

-Using a path-based routing algorithm

What are federated clusters?

Federated clusters in Kubernetes allow multiple Kubernetes clusters to be interconnected, forming a larger mesh of clusters. This allows for greater scale and redundancy, as well as simplified management of multiple clusters.

Federated clusters are configured by setting up a federated control plane, and then adding other Kubernetes clusters to the federated control plane. The federated control plane can be used to manage the other Kubernetes clusters in a number of ways, including:

  • The nodes in the other clusters
  • The Pods in the other clusters
  • The Services in the other clusters
  • The Secrets in the other clusters
  • The ConfigMaps in the other clusters
  • The Deployments in the other clusters
  • The ReplicationControllers in the other clusters
  • The Ingresses in the other clusters
  • The LoadBalancers in the other clusters

What is a Kubelet?

Kubelet is a daemon on each node that runs on each Kubernetes node. Kubelet is responsible for communicating with the API server to get information about the state of the nodes and pods in the cluster, and for pulling and pushing images to and from the nodes.

What is Kubectl?

Kubectl is a command-line interface for Kubernetes. With Kubectl, you can manage your Kubernetes clusters and applications. Kubectl can be used on your local machine, or you can use it with a Kubernetes cluster. kubectl can be used to create, delete, and manage Kubernetes objects.

What is Kube-proxy? 

Kube-proxy is a daemon that runs on each Kubernetes node. It is responsible for proxying pod IPs and service IPs to the correct pods and services.  Kube-proxy is started automatically by Kubernetes.  Kubernetes also uses kube-proxy to load balance services. 

What are “K8s”? 

k8s is an abbreviation for Kubernetes.

How are Kubernetes and Docker related?

Kubernetes is a platform for managing containers at scale, while Docker itself is a container technology that can be used by Kubernetes.

A container infrastructure, such as Docker, allows apps to be packaged into lightweight, portable, and self-sufficient units. Kubernetes is a platform for managing and orchestrating containers at scale. Along with Kubernetes, Docker gives you the ability to deploy and manage applications at large scales.

Conclusion

The interview process can be daunting, but by preparing for the most commonly asked questions and understanding the basics of what Kubernetes is and does, you’ll be well on your way to acing your interview. We wish you the best of luck in your upcoming interview!

RedwoodJS vs. BlitzJS: The Future of Fullstack JavaScript Meta-Frameworks

Redwood and Blitz are two up-and-coming full-stack meta-frameworks that provide tooling for creating SPAs, server-side rendered pages, and statically generated content, providing a CLI to generate end-to-end scaffolds. I’ve been waiting for a worthy Rails replacement in JavaScript since who-knows-when. This article is an overview of the two, and while I’ve given more breadth to Redwood (as it differs from Rails a great deal), I personally prefer Blitz.

As the post ended up being quite lengthy, below, we provide a comparison table for the hasty ones.

A bit of history first

If you started working as a web developer in the 2010s, you might not have even heard of Ruby on Rails, even though it gave us apps like Twitter, GitHub, Urban Dictionary, Airbnb, and Shopify. Compared to the web frameworks of its time, it was a breeze to work with. Rails broke the mold of web technologies by being a highly opinionated MVC tool, emphasizing the use of well-known patterns such as convention over configuration and DRY, with the addition of a powerful CLI that created end-to-end scaffolds from model to the template to be rendered. Many other frameworks have built on its ideas, such as Django for Python, Laravel for PHP, or Sails for Node.js. Thus, arguably, it is a piece of technology just as influential as the LAMP stack before its time.

However, the fame of Ruby on Rails has faded quite a bit since its creation in 2004. By the time I started working with Node.js in 2012, the glory days of Rails were over. Twitter — built on Rails — was infamous for frequently showcasing its fail whale between 2007 and 2009. Much of it was attributed to the lack of Rails’ scalability, at least according to word of mouth in my filter bubble. This Rails bashing was further reinforced when Twitter switched to Scala, even though they did not completely ditch Ruby then.

The scalability issues of Rails (and Django, for that matter) getting louder press coverage coincided with the transformation of the Web too. More and more JavaScript ran in the browser. Webpages became highly interactive WebApps, then SPAs. Angular.js revolutionized that too when it came out in 2010. Instead of the server rendering the whole webpage by combining the template and the data, we wanted to consume APIs and handle the state changes by client-side DOM updates.

Thus, full-stack frameworks fell out of favor. Development got separated between writing back-end APIs and front-end apps. And these apps could have meant Android and iOS apps too by that time, so it all made sense to ditch the server-side rendered HTML strings and send over the data in a way that all our clients could work with.

UX patterns developed as well. It wasn’t enough anymore to validate the data on the back-end, as users need quick feedback while they’re filling out bigger and bigger forms. Thus, our life got more and more complicated: we needed to duplicate the input validations and type definitions, even if we wrote JavaScript on both sides. The latter got simpler with the more widespread (re-)adoption of monorepos, as it got somewhat easier to share code across the whole system, even if it was built as a collection of microservices. But monorepos brought their own complications, not to mention distributed systems.

And ever since 2012, I have had a feeling that whatever problem we solve generates 20 new ones. You could argue that this is called “progress”, but maybe merely out of romanticism, or longing for times past when things used to be simpler, I’ve been waiting for a “Node.js on Rails” for a while now. Meteor seemed like it could be the one, but it quickly fell out of favor, as the community mostly viewed it as something that is good for MVPs but does not scale… The Rails problem all over again, but breaking down at an earlier stage of the product lifecycle. I must admit, I never even got around to try it.

However, it seemed like we were getting there slowly but steadily. Angular 2+ embraced the code generators á la Rails, alongside with Next.js, so it seemed like it could be something similar. Next.js got API Routes, making it possible to handle the front-end with SSR and write back-end APIs too. But it still lacks a powerful CLI generator and has nothing to do with the data layer either. And in general, a good ORM was still missing from the equation to reach the power level of Rails. At least this last point seems to be solved with Prisma being around now.

Wait a minute. We have code generators, mature back-end and front-end frameworks, and finally, a good ORM. Maybe we have all pieces of the puzzle in place? Maybe. But first, let’s venture a bit further from JavaScript and see if another ecosystem has managed to further the legacy of Rails, and whether we can learn from it.

Enter Elixir and Phoenix

Elixir is a language built on Erlang’s BEAM and OTP, providing a nice concurrency model based on the actor model and processes, which also results in easy error handling due to the “let it crash” philosophy in contrast to defensive programming. It also has a nice, Ruby-inspired syntax, yet remains to be an elegant, functional language.

Phoenix is built on top of Elixir’s capabilities, first as a simple reimplementation of Rails, with a powerful code generator, an data mapping toolkit (think ORM), good conventions, and generally good dev experience, with the inbuilt scalability of the OTP.

Yeah.. So far, I wouldn’t have even raised an eyebrow. Rails got more scalable over time, and I can get most of the things I need from a framework writing JavaScript these days, even if wiring it all up is still pretty much DIY. Anyhow, if I need an interactive browser app, I’ll need to use something like React (or at least Alpine.js) to do it anyway.

Boy, you can’t even start to imagine how wrong the previous statement is. While Phoenix is a full-fledged Rails reimplementation in Elixir, it has a cherry on top: your pages can be entirely server-side rendered and interactive at the same time, using its superpower called LiveView. When you request a LiveView page, the initial state gets prerendered on the server side, and then a WebSocket connection is built. The state is stored in memory on the server, and the client sends over events. The backend updates the state, calculates the diff, and sends over a highly compressed changeset to the UI, where a client-side JS library updates the DOM accordingly.

I heavily oversimplified what Phoenix is capable of, but this section is already getting too long, so make sure to check it out yourself!

We’ve taken a detour to look at one of the best, if not the best full-stack frameworks out there. So when it comes to full-stack JavaScript frameworks, it only makes sense to achieve at least what Phoenix has achieved. Thus, what I would want to see:

  1. A CLI that can generate data models or schemas, along with their controllers/services and their corresponding pages
  2. A powerful ORM like Prisma
  3. Server-side rendered but interactive pages, made simple
  4. Cross-platform usability: make it easy for me to create pages for the browser, but I want to be able to create an API endpoint responding with JSON by just adding a single line of code.
  5. Bundle this whole thing together

With that said, let’s see whether Redwood or Blitz is the framework we have been waiting for.

BlitzJS vs. RedwoodJS comparison

What is RedwoodJS?

Redwood markets itself as THE full-stack framework for startups. It is THE framework everyone has been waiting for, if not the best thing since the invention of sliced bread. End of story, this blog post is over.

At least according to their tutorial.

I felt a sort of boastful overconfidence while reading the docs, which I personally find difficult to read. The fact that it takes a lighter tone compared to the usual, dry, technical texts is a welcome change. Still, as a text moves away from the safe, objective description of things, it also wanders into the territory of matching or clashing with the reader’s taste. 

In my case, I admire the choice but could not enjoy the result.

Still, the tutorial is worth reading through. It is very thorough and helpful. The result is also worth the… well, whatever you feel while reading it, as Redwood is also nice to work with. Its code generator does what I would expect it to do. Actually, it does even more than I expected, as it is very handy not just for setting up the app skeleton, models, pages, and other scaffolds. It even sets your app up to be deployed to different deployment targets like AWS Lambdas, Render, Netlify, Vercel.

Speaking of the listed deployment targets, I have a feeling that Redwood pushes me a bit strongly towards serverless solutions, Render being the only one in the list where you have a constantly running service. And I like that idea too: if I have an opinionated framework, it sure can have its own opinions about how and where it wants to be deployed. As long as I’m free to disagree, of course.

But Redwood has STRONG opinions not just about the deployment, but overall on how web apps should be developed, and if you don’t agree with those, well…

I want you to use GraphQL

Let’s take a look at a freshly generated Redwood app. Redwood has its own starter kit, so we don’t need to install anything, and we can get straight to creating a skeleton.

$ yarn create redwood-app --ts ./my-redwood-app

You can omit the --ts flag if you want to use plain JavaScript instead.

Of course, you can immediately start up the development server and see that you got a nice UI already with yarn redwood dev. One thing to notice, which is quite commendable in my opinion, is that you don’t need to globally install a redwood CLI. Instead, it always remains project local, making collaboration easier.

Now, let’s see the directory structure.

my-redwood-app
├── api/
├── scripts/
├── web/
├── graphql.config.js
├── jest.config.js
├── node_modules
├── package.json
├── prettier.config.js
├── README.md
├── redwood.toml
├── test.js
└── yarn.lock

We can see the regular prettier.config.js, jest.config.js, and there’s also a redwood.toml for configuring the port of the dev-server. We have an api and web directory for separating the front-end and the back-end into their own paths using yarn workspaces.

But wait, we have a graphql.config.js too! That’s right, with Redwood, you’ll write a GraphQL API. Under the hood, Redwood uses Apollo on the front-end and Yoga on the back-end, but most of it is made pretty easy using the CLI. However, GraphQL has its downsides, and if you’re not OK with the tradeoff, well, you’re shit out of luck with Redwood.

Let’s dive a bit deeper into the API.

my-redwood-app
├── api
│   ├── db
│   │   └── schema.prisma
│   ├── jest.config.js
│   ├── package.json
│   ├── server.config.js
│   ├── src
│   │   ├── directives
│   │   │   ├── requireAuth
│   │   │   │   ├── requireAuth.test.ts
│   │   │   │   └── requireAuth.ts
│   │   │   └── skipAuth
│   │   │       ├── skipAuth.test.ts
│   │   │       └── skipAuth.ts
│   │   ├── functions
│   │   │   └── graphql.ts
│   │   ├── graphql
│   │   ├── lib
│   │   │   ├── auth.ts
│   │   │   ├── db.ts
│   │   │   └── logger.ts
│   │   └── services
│   ├── tsconfig.json
│   └── types
│       └── graphql.d.ts
...

Here, we can see some more, backend related config files, and the debut of tsconfig.json.

  • api/db/: Here resides our schema.prisma, which tells us the Redwood, of course, uses Prisma. The src/ dir stores the bulk of our logic.
  • directives/: Stores our graphql schema directives.
  • functions/: Here are the necessary lambda functions so we can deploy our app to a serverless cloud solution (remember STRONG opinions?).
  • graphql/: Here reside our gql schemas, which can be generated automatically from our db schema.
  • lib/: We can keep our more generic helper modules here.
  • services/: If we generate a page, we’ll have a services/ directory, which will hold our actual business logic.

This nicely maps to a layered architecture, where the GraphQL resolvers function as our controller layer. We have our services, and we can either create a repository or dal layer on top of Prisma, or if we can keep it simple, then use it as our data access tool straight away.

So far so good. Let’s move to the front-end.

my-redwood-app
├── web
│   ├── jest.config.js
│   ├── package.json
│   ├── public
│   │   ├── favicon.png
│   │   ├── README.md
│   │   └── robots.txt
│   ├── src
│   │   ├── App.tsx
│   │   ├── components
│   │   ├── index.css
│   │   ├── index.html
│   │   ├── layouts
│   │   ├── pages
│   │   │   ├── FatalErrorPage
│   │   │   │   └── FatalErrorPage.tsx
│   │   │   └── NotFoundPage
│   │   │       └── NotFoundPage.tsx
│   │   └── Routes.tsx
│   └── tsconfig.json
...

From the config file and the package.json, we can deduce we’re in a different workspace. The directory layout and file names also show us that this is not merely a repackaged Next.js app but something completely Redwood specific.

Redwood comes with its router, which is heavily inspired by React Router. I found this a bit annoying as the dir structure-based one in Next.js feels a lot more convenient, in my opinion.

However, a downside of Redwood is that it does not support server-side rendering, only static site generation. Right, SSR is its own can of worms, and while currently you probably want to avoid it even when using Next, with the introduction of Server Components this might soon change, and it will be interesting to see how Redwood will react (pun not intended).

On the other hand, Next.js is notorious for the hacky way you need to use layouts with it (which will soon change though), while Redwood handles them as you’d expect it. In Routes.tsx, you simply need to wrap your Routes in a Set block to tell Redwood what layout you want to use for a given route, and never think about it again.

import { Router, Route, Set } from "@redwoodjs/router";
import BlogLayout from "src/layouts/BlogLayout/";

const Routes = () => {
  return (
    <Router>
      <Route path="/login" page={LoginPage} name="login" />
      <Set wrap={BlogLayout}>
        <Route path="/article/{id:Int}" page={ArticlePage} name="article" />
        <Route path="/" page={HomePage} name="home" />
      </Set>
      <Route notfound page={NotFoundPage} />
    </Router>
  );
};

export default Routes;

Notice that you don’t need to import the page components, as it is handled automatically. Why can’t we also auto-import the layouts though, as for example Nuxt 3 would? Beats me.

Another thing to note is the /article/{id:Int} part. Gone are the days when you always need to make sure to convert your integer ids if you get them from a path variable, as Redwood can convert them automatically for you, given you provide the necessary type hint.

Now’s a good time to take a look at SSG. The NotFoundPage probably doesn’t have any dynamic content, so we can generate it statically. Just add prerender, and you’re good.

const Routes = () => {
  return (
    <Router>
      ...
      <Route notfound page={NotFoundPage} prerender />
    </Router>
  );
};

export default Routes;

You can also tell Redwood that some of your pages require authentication. Unauthenticated users should be redirected if they try to request it.

import { Private, Router, Route, Set } from "@redwoodjs/router";
import BlogLayout from "src/layouts/BlogLayout/";

const Routes = () => {
  return (
    <Router>
      <Route path="/login" page={LoginPage} name="login" />
      <Private unauthenticated="login">
        <Set wrap={PostsLayout}>
          <Route
            path="/admin/posts/new"
            page={PostNewPostPage}
            name="newPost"
          />
          <Route
            path="/admin/posts/{id:Int}/edit"
            page={PostEditPostPage}
            name="editPost"
          />
        </Set>
      </Private>
      <Set wrap={BlogLayout}>
        <Route path="/article/{id:Int}" page={ArticlePage} name="article" />
        <Route path="/" page={HomePage} name="home" />
      </Set>
      <Route notfound page={NotFoundPage} />
    </Router>
  );
};

export default Routes;

Of course, you need to protect your mutations and queries, too. So make sure to append them with the pre-generated @requireAuth.

Another nice thing in Redwood is that you might not want to use a local auth strategy but rather outsource the problem of user management to an authentication provider, like Auth0 or Netlify-Identity. Redwood’s CLI can install the necessary packages and generate the required boilerplate automatically.

What looks strange, however, at least with local auth, is that the client makes several roundtrips to the server to get the token. More specifically, the server will be hit for each currentUser or isAuthenticated call.

Frontend goodies in Redwood

There are two things that I really loved about working with Redwood: Cells and Forms.

A cell is a component that fetches and manages its own data and state. You define the queries and mutations it will use, and then export a function for rendering the Loading, Empty, Failure, and Success states of the component. Of course, you can use the generator to create the necessary boilerplate for you.

A generated cell looks like this:

import type { ArticlesQuery } from "types/graphql";
import type { CellSuccessProps, CellFailureProps } from "@redwoodjs/web";

export const QUERY = gql`
  query ArticlesQuery {
    articles {
      id
    }
  }
`;

export const Loading = () => <div>Loading...</div>;

export const Empty = () => <div>Empty</div>;

export const Failure = ({ error }: CellFailureProps) => (
  <div style={{ color: "red" }}>Error: {error.message}</div>
);

export const Success = ({ articles }: CellSuccessProps<ArticlesQuery>) => {
  return (
    <ul>
      {articles.map((item) => {
        return <li key={item.id}>{JSON.stringify(item)}</li>;
      })}
    </ul>
  );
};

Then you just import and use it as you would any other component, for example, on a page.

import ArticlesCell from "src/components/ArticlesCell";

const HomePage = () => {
  return (
    <>
      <MetaTags title="Home" description="Home page" />
      <ArticlesCell />
    </>
  );
};

export default HomePage;

However! If you use SSG on pages with cells — or any dynamic content really —only their loading state will get pre-rendered, which is not much of a help. That’s right, no getStaticProps for you if you go with Redwood.

The other somewhat nice thing about Redwood is the way it eases form handling, though the way they frame it leaves a bit of a bad taste in my mouth. But first, the pretty part.

import { Form, FieldError, Label, TextField } from "@redwoodjs/forms";

const ContactPage = () => {
  return (
    <>
      <Form config={{ mode: "onBlur" }}>
        <Label name="email" errorClassName="error">
          Email
        </Label>
        <TextField
          name="email"
          validation={{
            required: true,
            pattern: {
              value: /^[^@]+@[^.]+\..+$/,
              message: "Please enter a valid email address",
            },
          }}
          errorClassName="error"
        />
        <FieldError name="email" className="error" />
      </Form>
    </>
  );
};

The TextField components validation attribute expects an object to be passed, with a pattern against which the provided input value can be validated.

The errorClassName makes it easy to set the style of the text field and its label in case the validation fails, e.g. turning it red. The validations message will be printed in the FieldError component. Finally, the config={{ mode: 'onBlur' }} tells the form to validate each field when the user leaves them.

The only thing that spoils the joy is the fact that this pattern is eerily similar to the one provided by Phoenix. Don’t get me wrong. It is perfectly fine, even virtuous, to copy what’s good in other frameworks. But I got used to paying homage when it’s due. Of course, it’s totally possible that the author of the tutorial did not know about the source of inspiration for this pattern. If that’s the case, let me know, and I’m happy to open a pull request to the docs, adding that short little sentence of courtesy.

But let’s continue and take a look at the whole working form.

import { MetaTags, useMutation } from "@redwoodjs/web";
import { toast, Toaster } from "@redwoodjs/web/toast";
import {
  FieldError,
  Form,
  FormError,
  Label,
  Submit,
  SubmitHandler,
  TextAreaField,
  TextField,
  useForm,
} from "@redwoodjs/forms";

import {
  CreateContactMutation,
  CreateContactMutationVariables,
} from "types/graphql";

const CREATE_CONTACT = gql`
  mutation CreateContactMutation($input: CreateContactInput!) {
    createContact(input: $input) {
      id
    }
  }
`;

interface FormValues {
  name: string;
  email: string;
  message: string;
}

const ContactPage = () => {
  const formMethods = useForm();

  const [create, { loading, error }] = useMutation<
    CreateContactMutation,
    CreateContactMutationVariables
  >(CREATE_CONTACT, {
    onCompleted: () => {
      toast.success("Thank you for your submission!");
      formMethods.reset();
    },
  });

  const onSubmit: SubmitHandler<FormValues> = (data) => {
    create({ variables: { input: data } });
  };

  return (
    <>
      <MetaTags title="Contact" description="Contact page" />

      <Toaster />
      <Form
        onSubmit={onSubmit}
        config={{ mode: "onBlur" }}
        error={error}
        formMethods={formMethods}
      >
        <FormError error={error} wrapperClassName="form-error" />

        <Label name="email" errorClassName="error">
          Email
        </Label>
        <TextField
          name="email"
          validation={{
            required: true,
            pattern: {
              value: /^[^@]+@[^.]+\..+$/,
              message: "Please enter a valid email address",
            },
          }}
          errorClassName="error"
        />
        <FieldError name="email" className="error" />

        <Submit disabled={loading}>Save</Submit>
      </Form>
    </>
  );
};

export default ContactPage;

Yeah, that’s quite a mouthful. But this whole thing is necessary if we want to properly handle submissions and errors returned from the server. We won’t dive deeper into it now, but if you’re interested, make sure to take a look at Redwood’s really nicely written and thorough tutorial.

Now compare this with how it would look like in Phoenix LiveView.

<div>
  <.form
    let={f}
    for={@changeset}
    id="contact-form"
    phx-target={@myself}
    phx-change="validate"
    phx-submit="save">

    <%= label f, :title %>
    <%= text_input f, :title %>
    <%= error_tag f, :title %>

    <div>
      <button type="submit" phx-disable-with="Saving...">Save</button>
    </div>
  </.form>
</div>

A lot easier to see through while providing almost the same functionality. Yes, you’d be right to call me out for comparing apples to oranges. One is a template language, while the other is JSX. Much of the logic in a LiveView happens in an elixir file instead of the template, while JSX is all about combining the logic with the view. However, I’d argue that an ideal full-stack framework should allow me to write the validation code once for inputs, then let me simply provide the slots in the view to insert the error messages into, and allow me to set up the conditional styles for invalid inputs and be done with it. This would provide a way to write cleaner code on the front-end, even when using JSX. You could say this is against the original philosophy of React, and my argument merely shows I have a beef with it. And you’d probably be right to do so. But this is an opinion article about opinionated frameworks, after all, so that’s that.

The people behind RedwoodJS

Credit, where credit is due.

Redwood was created by GitHub co-founder and former CEO Tom Preston-Werner, Peter Pistorius, David Price & Rob Cameron. Moreover, its core team currently consists of 23 people. So if you’re afraid to try out newish tools because you may never know when their sole maintainer gets tired of the struggles of working on a FOSS tool in their free time, you can rest assured: Redwood is here to stay.

Redwood: Honorable mentions

Redwood

  • also comes bundled with Storybook,
  • provides the must-have graphiql-like GraphQL Playground,
  • provides accessibility features out of the box like the RouteAnnouncemnet SkipNavLink, SkipNavContent and RouteFocus components,
  • of course it automatically splits your code by pages.

The last one is somewhat expected in 2022, while the accessibility features would deserve their own post in general. Still, this one is getting too long already, and we haven’t even mentioned the other contender yet.

Let’s see BlitzJS

Blitz is built on top of Next.js, and it is inspired by Ruby on Rails and provides a “Zero-API” data layer abstraction. No GraphQL, pays homage to predecessors… seems like we’re off to a good start. But does it live up to my high hopes? Sort of.

A troubled past

Compared to Redwood, Blitz’s tutorial and documentation are a lot less thorough and polished. It also lacks several convenience features: 

  • It does not really autogenerate host-specific config files.
  • Blitz cannot run a simple CLI command to set up auth providers.
  • It does not provide accessibility helpers.
  • Its code generator does not take into account the model when generating pages.

Blitz’s initial commit was made in February 2020, a bit more than half a year after Redwood’s in June 2019, and while Redwood has a sizable number of contributors, Blitz’s core team consists of merely 2-4 people. In light of all this, I think they deserve praise for their work.

But that’s not all. If you open up their docs, you’ll be greeted with a banner on top announcing a pivot.

While Blitz originally included Next.js and was built around it, Brandon Bayer and the other developers felt it was too limiting. Thus they forked it, which turned out to be a pretty misguided decision. It quickly became obvious that maintaining the fork would take a lot more effort than the team could invest.

All is not lost, however. The pivot aims to turn the initial value proposition “JavaScript on Rails with Next” into “JavaScript on Rails, bring your own Front-end Framework”. 

And I can’t tell you how relieved I am that this recreation of Rails won’t force me to use React. 

Don’t get me wrong. I love the inventiveness that React brought to the table. Front-end development has come a long way in the last nine years, thanks to React. Other frameworks like Vue and Svelte might lack behind in following the new concepts, but this also means they have more time to polish those ideas even further and provide better DevX. Or at least I find them a lot easier to work with without ever being afraid that my client-side code’s performance would grind to a standstill.

All in all, I find this turn of events a lucky blunder.

How to create a Blitz app

You’ll need to install Blitz globally (run yarn global add blitz or npm install -g blitz –legacy-peer-deps), before you create a Blitz app. That’s possibly my main woe when it comes to Blitz’s design, as this way, you cannot lock your project across all contributors to use a given Blitz CLI version and increment it when you see fit, as Blitz will automatically update itself from time to time.

Once blitz is installed, run

$ blitz new my-blitz-app

It will ask you 

  • whether you want to use TS or JS, 
  • if it should include a DB and Auth template (more on that later), 
  • if you want to use npm, yarn or pnpm to install dependencies, 
  • and if you want to use React Final Form or React Hook Form. 

Once you have answered all its questions, the CLI starts to download half of the internet, as it is customary. Grab something to drink, have a lunch, finish your workout session, or whatever you do to pass the time and when you’re done, you can fire up the server by running

$ blitz dev

And, of course, you’ll see the app running and the UI telling you to run

$ blitz generate all project name:string

But before we do that, let’s look around in the project directory.

my-blitz-app/
├── app/
├── db/
├── mailers/
├── node_modules/
├── public/
├── test/
├── integrations/
├── babel.config.js
├── blitz.config.ts
├── blitz-env.d.ts
├── jest.config.ts
├── package.json
├── README.md
├── tsconfig.json
├── types.ts
└── yarn.lock

Again, we can see the usual suspects: config files, node_modules, test, and the likes. The public directory — to no one’s surprise — is the place where you store your static assets. Test holds your test setup and utils. Integrations is for configuring your external services, like a payment provider or a mailer. Speaking of the mailer, that is where you can handle your mail-sending logic. Blitz generates a nice template with informative comments for you to get started, including a forgotten password email template.

As you’d probably guessed, the app and db directories are the ones where you have the bulk of your app-related code. Now’s the time to do as the generated landing page says and run blitz generate all project name:string.

Say yes, when it asks you if you want to migrate your database and give it a descriptive name like add project.

Now let’s look at the db directory.

my-blitz-app/
└── db/
    ├── db.sqlite
    ├── db.sqlite-journal
    ├── index.ts
    ├── migrations/
    │   ├── 20220610075814_initial_migration/
    │   │   └── migration.sql
    │   ├── 20220610092949_add_project/
    │   │   └── migration.sql
    │   └── migration_lock.toml
    ├── schema.prisma
    └── seeds.ts

The migrations directory is handled by Prisma, so it won’t surprise you if you’re already familiar with it. If not, I highly suggest trying it out on its own before you jump into using either Blitz or Redwood, as they heavily and transparently rely on it.

Just like in Redwood’s db dir, we have our schema.prisma, and our sqlite db, so we have something to start out with. But we also have a seeds.ts and index.ts. If you take a look at the index.ts file, it merely re-exports Prisma with some enhancements, while the seeds.ts file kind of speaks for itself.

Now’s the time to take a closer look at our schema.prisma.

// This is your Prisma schema file,
// learn more about it in the docs: https://pris.ly/d/prisma-schema

datasource db {
  provider = "sqlite"
  url      = env("DATABASE_URL")
}

generator client {
  provider = "prisma-client-js"
}

// --------------------------------------

model User {
  id             Int      @id @default(autoincrement())
  createdAt      DateTime @default(now())
  updatedAt      DateTime @updatedAt
  name           String?
  email          String   @unique
  hashedPassword String?
  role           String   @default("USER")

  tokens   Token[]
  sessions Session[]
}

model Session {
  id                 Int       @id @default(autoincrement())
  createdAt          DateTime  @default(now())
  updatedAt          DateTime  @updatedAt
  expiresAt          DateTime?
  handle             String    @unique
  hashedSessionToken String?
  antiCSRFToken      String?
  publicData         String?
  privateData        String?

  user   User? @relation(fields: [userId], references: [id])
  userId Int?
}

model Token {
  id          Int      @id @default(autoincrement())
  createdAt   DateTime @default(now())
  updatedAt   DateTime @updatedAt
  hashedToken String
  type        String
  // See note below about TokenType enum
  // type        TokenType
  expiresAt   DateTime
  sentTo      String

  user   User @relation(fields: [userId], references: [id])
  userId Int

  @@unique([hashedToken, type])
}

// NOTE: It's highly recommended to use an enum for the token type
//       but enums only work in Postgres.
//       See: https://blitzjs.com/docs/database-overview#switch-to-postgre-sql
// enum TokenType {
//   RESET_PASSWORD
// }

model Project {
  id        Int      @id @default(autoincrement())
  createdAt DateTime @default(now())
  updatedAt DateTime @updatedAt
  name      String
}

As you can see, Blitz starts out with models to be used with a fully functional User management. Of course, it also provides all the necessary code in the app scaffold, meaning that the least amount of logic is abstracted away, and you are free to modify it as you see fit.

Below all the user-related models, we can see the Project model we created with the CLI, with an automatically added id, createdAt, and updatedAt files. One of the things that I prefer in Blitz over Redwood is that its CLI mimics Phoenix, and you can really create everything from the command line end-to-end. 

This really makes it easy to move quickly, as less context switching happens between the code and the command line. Well, it would if it actually worked, as while you can generate the schema properly, the generated pages, mutations, and queries always use name: string, and disregard the entity type defined by the schema, unlike Redwood. There’s already an open pull request to fix this, but the Blitz team understandably has been focusing on getting v2.0 done instead of patching up the current stable branch.

That’s it for the db, let’s move on to the app directory.

my-blitz-app
└── app
    ├── api/
    ├── auth/
    ├── core/
    ├── pages/
    ├── projects/
    └── users/

The core directory contains Blitz goodies, like a predefined and parameterized Form (without Redwood’s or Phoenix’s niceties though), a useCurrentUser hook, and a Layouts directory, as Bliz made it easy to persist layouts between pages, which will be rendered completely unnecessary with the upcoming Next.js Layouts. This reinforces further that the decision to ditch the fork and pivot to a toolkit was probably a difficult but necessary decision.

The auth directory contains the fully functional authentication logic we talked about earlier, with all the necessary database mutations such as signup, login, logout, and forgotten password, with their corresponding pages and a signup and login form component. The getCurrentUser query got its own place in the users directory all by itself, which makes perfect sense.

And we got to the pages and projects directories, where all the action happens.

Blitz creates a directory to store database queries, mutations, input validations (using zod), and model-specific components like create and update forms in one place. You will need to fiddle around in these a lot, as you will need to update them according to your actual model. This is nicely laid out though in the tutorial… Be sure to read it, unlike I did when I first tried Blitz out.

my-blitz-app/
└── app/
    └── projects/
        ├── components/
        │   └── ProjectForm.tsx
        ├── mutations/
        │   ├── createProject.ts
        │   ├── deleteProject.ts
        │   └── updateProject.ts
        └── queries/
            ├── getProjects.ts
            └── getProject.ts

Whereas the pages directory won’t be of any surprise if you’re already familiar with Next.

my-blitz-app/
└── app/
    └── pages/
        ├── projects/
        │   ├── index.tsx
        │   ├── new.tsx
        │   ├── [projectId]/
        │   │   └── edit.tsx
        │   └── [projectId].tsx
        ├── 404.tsx
        ├── _app.tsx
        ├── _document.tsx
        ├── index.test.tsx
        └── index.tsx

A bit of explanation if you haven’t tried Next out yet: Blitz uses file-system-based routing just like Next. The pages directory is your root, and the index file is rendered when the path corresponding to a given directory is accessed. Thus when the root path is requested, pages/index.tsx will be rendered, accessing /projects will render pages/projects/index.tsx, /projects/new will render pages/projects/new.tsx and so on. 

If a filename is enclosed in []-s, it means that it corresponds to a route param. Thus /projects/15 will render pages/projects/[projectId].tsx. Unlike in Next, you access the param’s value within the page using the <code>useParam(name: string, type?: string)</code> hook. To access the query object, use the <code>useRouterQuery(name: string)</code>. To be honest, I never really understood why Next needs to mesh together the two.

When you generate pages using the CLI, all pages are protected by default. To make them public, simply delete the [PageComponent].authenticate = true line. This will throw an AuthenticationError if the user is not logged in anyway, so if you’d rather redirect unauthenticated users to your login page, you probably want to use [PageComponent].authenticate = {redirectTo: '/login'}.

In your queries and mutations, you can use the ctx context arguments value to call ctx.session.$authorize or resolver.authorize in a pipeline to secure your data.

Finally, if you still need a proper http API, you can create Express-style handler functions, using the same file-system routing as for your pages.

A possible bright future

While Blitz had a troubled past, it might have a bright future. It is still definitely in the making and not ready for widespread adoption. The idea of creating a framework agnostic full-stack JavaScript toolkit is a versatile concept. This strong concept is further reinforced by the good starting point, which is the current stable version of Blitz. I’m looking further to see how the toolkit will evolve over time.

Redwood vs. Blitz: Comparison and Conclusion

I set out to see whether we have a Rails, or even better, Phoenix equivalent in JavaScript. Let’s see how they measured up.

1. CLI code generator

Redwood’s CLI gets the checkmark on this one, as it is versatile, and does what it needs to do. The only small drawback is that the model has to be written in file first, and cannot be generated.

Blitz’s CLI is still in the making, but that’s true about Blitz in general, so it’s not fair to judge it by what’s ready, but only by what it will be. In that sense, Blitz would win if it was fully functional (or will when it will be), as it can really generate pages end-to-end.

Verdict: Tie

2. A powerful ORM

That’s a short one. Both use Prisma, which is a powerful enough ORM.

Verdict: Tie

3. Server side rendered but interactive pages

Well, in today’s ecosystem, that might be wishful thinking. Even in Next, SSR is something you should avoid, at least until we’ll have Server Components in React.

But which one mimics this behavior the best?

Redwood does not try to look like a Rails replacement. It has clear boundaries demarcated by yarn workspaces between front-end and back-end . It definitely provides nice conventions and — to keep it charitable — nicely reinvented the right parts of Phoenix’s form handling. However, strictly relying on GraphQL feels a bit overkill. For small apps that we start out with anyway when opting to use a full-stack framework, it definitely feels awkward.

Redwood is also React exclusive, so if you prefer using Vue, Svelte or Solid, then you have to wait until someone reimplements Redwood for your favorite framework.

Blitz follows the Rails way, but the controller layer is a bit more abstract. This is understandable, though, as using Next’s file-system-based routing, a lot of things that made sense for Rails do not make sense for Blitz. And in general, it feels more natural than using GraphQL for everything. In the meantime, becoming framework agnostic makes it even more versatile than Redwood.

Moreover, Blitz is on its way to becoming framework agnostic, so even if you’d never touch React, you’ll probably be able to see its benefits in the near future.

But to honor the original criterion: Redwood provides client-side rendering and SSG (kind of), while Blitz provides SSR on top of the previous two.

Verdict: Die-hard GraphQL fans will probably want to stick with Redwood. But according to my criteria, Blitz hands down wins this one.

4. API

Blitz auto generates an API for data access that you can use if you want to, but you can explicitly write handler functions too. A little bit awkward, but the possibility is there.

Redwood maintains a hard separation between front-end and back-end, so it is trivial that you have an API, to begin with. Even if it’s a GraphQL API, that might just be way too much to engineer for your needs.

Verdict: Tie (TBH, I feel like they both suck at this the same amount.)

Bye now!

In summary, Redwood is a production-ready, React+GraphQL-based full-stack JavaScript framework made for the edge. It does not follow the patterns laid down by Rails at all, except for being highly opinionated. It is a great tool to use if you share its sentiment, but my opinion greatly differs from Redwood’s on what makes development effective and enjoyable.

Blitz, on the other hand, follows in the footsteps of Rails and Next, and is becoming a framework agnostic, full-stack toolkit that eliminates the need for an API layer.

I hope you found this comparison helpful. Leave a comment if you agree with my conclusion and share my love for Blitz. If you don’t, argue with the enlightened ones… they say controversy boosts visitor numbers.

Argo CD Kubernetes Tutorial

Usually, when devs set up a CI/CD pipeline for an application hosted on Kubernetes, they handle both the CI and CD parts in one task runner, such as CircleCI or Travis CI. These services offer push-based updates to your deployments, which means that credentials for the code repo and the deployment target must be stored with these services. This method can be problematic if the service gets compromised, e.g. as it happened to CodeShip.

Even using services such as GitLab CI and GitHub Actions requires that credentials for accessing your cluster be stored with them. If you’re employing GitOps, to take advantage of using the usual Push to repo -> Review Code -> Merge Code sequence for managing your infrastructure configuration as well, this would also mean access to your whole infrastructure.

[elementor-template id="3483"]

Luckily there are tools to help us with these issues. Two of the most known are Argo CD and Flux. They allow credentials to be stored within your Kubernetes cluster, where you have more control over their security. They also offer pull-based deployment with drift detection. Both of these tools solve the same issues, but tackle them from different angles.

Here, we’ll take a deeper look at Argo CD out of the two.

What is Argo CD

Argo CD is a continuous deployment tool that you can install into your Kubernetes cluster. It can pull the latest code from a git repository and deploy it into the cluster – as opposed to external CD services, deployments are pull-based. You can manage updates for both your application and infrastructure configuration with Argo CD. Advantages of such a setup include being able to use credentials from the cluster itself for deployments, which can be stored in secrets or a vault.

Preparation

To try out Argo CD, we’ve also prepared a test project that we’ll deploy to Kubernetes hosted on DigitalOcean. You can grab the example project from our GitLab repository here: https://gitlab.com/risingstack-org/argocd-demo/

Forking the repo will allow you to make changes for yourself, and it can be set up later in Argo CD as the deployment source.

Get doctl from here:

https://docs.digitalocean.com/reference/doctl/how-to/install/

Or, if you’re using a mac, from Homebrew:

brew install doctl

You can use any Kubernetes provider for this tutorial. The two requirements are having a Docker repository and a Kubernetes cluster with access to it. For this tutorial, we chose to go with DigitalOcean for the simplicity of its setup, but most other platforms should work just fine.

We’ll focus on using the web UI for the majority of the process, but you can also opt to use the `doctl` cli tool if you wish. `doctl` can mostly replace `kubectl` as well. `doctl` will only be needed to push our built docker image to the repo that our deployment will have access to.

Helm is a templating engine for Kubernetes. It allows us to define values separately from the structure of the yaml files, which can help with access control and managing multiple environments using the same template.

You can grab Helm here: https://github.com/helm/helm/releases

Or via Homebrew for mac users:

brew install helm

Download the latest Argo CD version from https://github.com/argoproj/argo-cd/releases/latest

If you’re using a mac, you can grab the cli tools from Homebrew:

brew install argocd

DigitalOcean Setup

After logging in, first, create a cluster using the “Create” button on the top right, and selecting Kubernetes. For the purposes of this demo, we can just go with the smallest cluster with no additional nodes. Be sure to choose a data center close to you.

Preparing the demo app

You can find the demo app in the node-app folder in the repo you forked. Use this folder for the following steps to build and push the docker image to the GitLab registry:

docker login registry.gitlab.com

docker build . -t registry.gitlab.com/<substiture repo name here>/demo-app-1

docker push registry.gitlab.com/<substiture repo name here>/demo-app-1

GitLab offers a free image registry with every git repo – even free tier ones. You can use these to store your built image, but be aware that the registry inherits the privacy setting of the git repo, you can’t change them separately.

Once the image is ready, be sure to update the values.yaml file with the correct image url and use helm to generate the resources.yaml file. You can then deploy everything using kubectl:

helm template -f "./helm/demo-app/values.yaml" "./helm/demo-app" > "./helm/demo-app/resources/resources.yaml"

kubectl apply -f helm/demo-app/resources/resources.yaml

The only purpose of these demo-app resources’ is to showcase the ArgoCD UI capabilities, that’s why it also contains an Ingress resource as a plus.

Install Argo CD into the cluster

Argo CD provides a yaml file that installs everything you’ll need and it’s available online. The most important thing here is to make sure that you install it into the `argocd` namespace, otherwise, you’ll run into some errors later and Argo CD will not be usable.

kubectl create namespace argocd

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

From here, you can use Kubernetes port-forwarding to access the UI of Argo CD:

kubectl -n argocd port-forward svc/argocd-server 8080:443

This will expose the service on localhost:8080 – we will use the UI to set up the connection to GitLab, but it could also be done via the command line tool.

Argo CD setup

To log in on the UI, use `admin` as username, and the password retrieved by this command:

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

Once you’re logged in, connect your fork of the demo app repo from the Repositories inside the Settings menu on the left side. Here, we can choose between ssh and https authentication – for this demo, we’ll use https, but for ssh, you’d only need to set up a key pair for use.

argo cd repo connect

Create an API key on GitLab and use it in place of a password alongside your username to connect the repo. An API key allows for some measure of access control as opposed to using your account password.

After successfully connecting the repository, the only thing left is to set up an Application, which will take care of synchronizing the state of our deployment with that described in the GitLab repo.

argo cd tutorial how to set up a new application

You’ll need to choose a branch or a tag to use to monitor. Let’s choose the master branch for now – it should contain the latest stable code anyway. Setting the sync policy to automatic allows for automatic deployments when the git repo is updated, and also provides automatic pruning and self-healing capabilities.

argo cd application setup

Be sure to set the destination cluster to the one available in the dropdown and use the `demo` namespace. If everything is set correctly, Argo CD should now start syncing the deployment state.

Features of Argo CD

From the application view, you can now see the different parts that comprise our demo application.

argo cd app overview

Clicking on any of these parts allows for checking the diff of the deployed config, and the one checked into git, as well as the yaml files themselves separately. The diff should be empty for now, but we’ll see it in action once we make some changes or if you disable automatic syncing.

argo cd container details

You also have access to the logs from the pods here, which can be quite useful – logs are not retained between different pod instances, which means that they are lost on the deletion of a pod, however.

argo cd container logs

It is also possible to handle rollbacks from here, clicking on the “History and Rollback” button. Here, you can see all the different versions that have been deployed to our cluster by commit. 

You can re-deploy any of them using the … menu on the top right, and selecting “Redeploy” – this feature needs automatic deployment to be turned off. However, you’ll be prompted to do so here.

These should cover the most important parts of the UI and what is available in Argo CD. Next up, we’ll take a look at how the deployment update happens when code changes on GitLab.

Updating the deployment

With the setup done, any changes you make to the configuration that you push to the master branch should be reflected on the deployment shortly after.

A very simple way to check out the updating process is to bump up the `replicaCount` in values.yaml to 2 (or more), and run the helm command again to generate the resources.yaml. 

Then, commit and push to master and monitor the update process on the Argo CD UI.

You should see a new event in the demo-app events, with the reason `ScalingReplicaSet`.

argo cd scaling event

You can double-check the result using kubectl, where you should now see two instances of the demo-app running:

kubectl -n demo get pod

There is another branch prepared in the repo, called second-app, which has another app that you can deploy, so you can see some more of the update process and diffs. It is quite similar to how the previous deployment works.

First, you’ll need to merge the second-app branch into master – this will allow the changes to be automatically deployed, as we set it up already. Then, from the node-app-2 folder, build and push the docker image. Make sure to have a different version tag for it, so we can use the same repo!

docker build . -t registry.gitlab.com/<substitute repo name here>/demo-app-2

docker push registry.gitlab.com/<substitute repo name here>/demo-app-2

You can set deployments to manual for this step, to be able to take a better look at the diff before the actual update happens. You can do this from the sync settings part of `App details`.

argo cd sync policy

Generate the updated resources file afterwards, then commit and push it to git to trigger the update in Argo CD:

helm template -f "./helm/demo-app/values.yaml" "./helm/demo-app" > "./helm/demo-app/resources/resources.yaml"

This should result in a diff appearing `App details` -> `Diff` for you to check out. You can either deploy it manually or just turn auto-deploy back.

ArgoCD safeguards you from those resource changes that are drifting from the latest source-controlled version of your code. Let’s try to manually scale up the deployment to 5 instances:

Get the name of the replica set:

kubectl -n demo get rs

Scale it to 5 instances:

kubectl -n demo scale --replicas=5 rs/demo-app-<number>

If you are quick enough, you can catch the changes applied on the ArgoCD Application Visualization as it tries to add those instances. However, ArgoCD will prevent this change, because it would drift from the source controlled version of the deployment. It also scales the deployment down to the defined value in the latest commit (in my example it was set to 3). 

The downscale event can be found under the `demo-app` deployment events, as shown below:

how to scale down kubernetes with argo cd

From here, you can experiment with whatever changes you’d like!

Finishing our ArgoCD Kubernetes Tutorial

This was our quick introduction to using ArgoCD, which can make your GitOps workflow safer and more convenient.

Stay tuned, as we’re planning to take a look at the other heavy-hitter next time: Flux.

This article was written by Janos Kubisch, senior engineer at RisingStack.

How to Deploy a Ceph Storage to Bare Virtual Machines

Ceph is a freely available storage platform that implements object storage on a single distributed computer cluster and provides interfaces for object-, block- and file-level storage. Ceph aims primarily for completely distributed operation without a single point of failure. Ceph storage manages data replication and is generally quite fault-tolerant. As a result of its design, the system is both self-healing and self-managing.

Ceph has loads of benefits and great features, but the main drawback is that you have to host and manage it yourself. In this post, we’ll check two different approaches of virtual machine deployment with Ceph.

Anatomy of a Ceph cluster

Before we dive into the actual deployment process, let’s see what we’ll need to fire up for our own Ceph cluster.

There are three services that form the backbone of the cluster

  • ceph monitors (ceph-mon) maintain maps of the cluster state and are also responsible for managing authentication between daemons and clients
  • managers (ceph-mgr) are responsible for keeping track of runtime metrics and the current state of the Ceph cluster
  • object storage daemons (ceph-osd) store data, handle data replication, recovery, rebalancing, and provide some ceph monitoring information.

Additionally, we can add further parts to the cluster to support different storage solutions

  • metadata servers (ceph-mds) store metadata on behalf of the Ceph Filesystem
  • rados gateway (ceph-rgw) is an HTTP server for interacting with a Ceph Storage Cluster that provides interfaces compatible with OpenStack Swift and Amazon S3.

There are multiple ways of deploying these services. We’ll check two of them:

  • first, using the ceph/deploy tool,
  • then a docker-swarm based vm deployment.

Let’s kick it off!

Ceph Setup

Okay, a disclaimer first. As this is not a production infrastructure, we’ll cut a couple of corners.

You should not run multiple different Ceph demons on the same host, but for the sake of simplicity, we’ll only use 3 virtual machines for the whole cluster.

In the case of OSDs, you can run multiple of them on the same host, but using the same storage drive for multiple instances is a bad idea as the disk’s I/O speed might limit the OSD daemons’ performance.

For this tutorial, I’ve created 4 EC2 machines in AWS: 3 for Ceph itself and 1 admin node. For ceph-deploy to work, the admin node requires passwordless SSH access to the nodes and that SSH user has to have passwordless sudo privileges.

In my case, as all machines are in the same subnet on AWS, connectivity between them is not an issue. However, in other cases editing the hosts file might be necessary to ensure proper connection.

Depending on where you deploy Ceph security groups, firewall settings or other resources have to be adjusted to open these ports

  • 22 for SSH
  • 6789 for monitors
  • 6800:7300 for OSDs, managers and metadata servers
  • 8080 for dashboard
  • 7480 for rados gateway

Without further ado, let’s start deployment.

Ceph Storage Deployment

Install prerequisites on all machines

$ sudo apt update
$ sudo apt -y install ntp python

For Ceph to work seamlessly, we have to make sure the system clocks are not skewed. The suggested solution is to install ntp on all machines and it will take care of the problem. While we’re at it, let’s install python on all hosts as ceph-deploy depends on it being available on the target machines.

Prepare the admin node

$ ssh -i ~/.ssh/id_rsa -A ubuntu@13.53.36.123

As all the machines have my public key added to known_hosts thanks to AWS, I can use ssh agent forwarding to access the Ceph machines from the admin node. The first line ensures that my local ssh agent has the proper key in use and the -A flag takes care of forwarding my key.

$ wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add -
echo deb https://download.ceph.com/debian-nautilus/ $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.list
$ sudo apt update
$ sudo apt -y install ceph-deploy

We’ll use the latest nautilus release in this example. If you want to deploy a different version, just change the debian-nautilus part to your desired release (luminous, mimic, etc.).

$ echo "StrictHostKeyChecking no" | sudo tee -a /etc/ssh/ssh_config > /dev/null

OR

$ ssh-keyscan -H 10.0.0.124,10.0.0.216,10.0.0.104 >> ~/.ssh/known_hosts

Ceph-deploy uses SSH connections to manage the nodes we provide. Each time you SSH to a machine that is not in the list of known_hosts (~/.ssh/known_hosts), you’ll get prompted whether you want to continue connecting or not. This interruption does not mesh well with the deployment process, so we either have to use ssh-keyscan to grab the fingerprint of all the target machines or disable the strict host key checking outright.

10.0.0.124 ip-10-0-0-124.eu-north-1.compute.internal ip-10-0-0-124
10.0.0.216 ip-10-0-0-216.eu-north-1.compute.internal ip-10-0-0-216
10.0.0.104 ip-10-0-0-104.eu-north-1.compute.internal ip-10-0-0-104

Even though the target machines are in the same subnet as our admin and they can access each other, we have to add them to the hosts file (/etc/hosts) for ceph-deploy to work properly. Ceph-deploy creates monitors by the provided hostname, so make sure it matches the actual hostname of the machines otherwise the monitors won’t be able to join the quorum and the deployment fails. Don’t forget to reboot the admin node for the changes to take effect.

$ mkdir ceph-deploy
$ cd ceph-deploy

As a final step of the preparation, let’s create a dedicated folder as ceph-deploy will create multiple config and key files during the process.

Deploy resources

$ ceph-deploy new ip-10-0-0-124 ip-10-0-0-216 ip-10-0-0-104

The command ceph-deploy new creates the necessary files for the deployment. Pass it the hostnames of the monitor nodes, and it will create cepf.conf and ceph.mon.keyring along with a log file.

The ceph-conf should look something like this

[global]
fsid = 0572e283-306a-49df-a134-4409ac3f11da
mon_initial_members = ip-10-0-0-124, ip-10-0-0-216, ip-10-0-0-104
mon_host = 10.0.0.124,10.0.0.216,10.0.0.104
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx

It has a unique ID called fsid, the monitor hostnames and addresses and the authentication modes. Ceph provides two authentication modes: none (anyone can access data without authentication) or cephx (key based authentication).

The other file, the monitor keyring is another important piece of the puzzle, as all monitors must have identical keyrings in a cluster with multiple monitors. Luckily ceph-deploy takes care of the propagation of the key file during virtual deployments.

$ ceph-deploy install --release nautilus ip-10-0-0-124 ip-10-0-0-216 ip-10-0-0-104

As you might have noticed so far, we haven’t installed ceph on the target nodes yet. We could do that one-by-one, but a more convenient way is to let ceph-deploy take care of the task. Don’t forget to specify the release of your choice, otherwise you might run into a mismatch between your admin and targets.

$ ceph-deploy mon create-initial

Finally, the first piece of the cluster is up and running! create-initial will deploy the monitors specified in ceph.conf we generated previously and also gather various key files. The command will only complete successfully if all the monitors are up and in the quorum.

$ ceph-deploy admin ip-10-0-0-124 ip-10-0-0-216 ip-10-0-0-104

Executing ceph-deploy admin will push a Ceph configuration file and the ceph.client.admin.keyring to the /etc/ceph directory of the nodes, so we can use the ceph CLI without having to provide the ceph.client.admin.keyring each time to execute a command.

At this point, we can take a peek at our cluster. Let’s SSH into a target machine (we can do it directly from the admin node thanks to agent forwarding) and run sudo ceph status.

$ sudo ceph status
  cluster:
	id: 	0572e283-306a-49df-a134-4409ac3f11da
	health: HEALTH_OK

  services:
	mon: 3 daemons, quorum ip-10-0-0-104,ip-10-0-0-124,ip-10-0-0-216 (age 110m)
mgr: no daemons active
osd: 0 osds: 0 up, 0 in

  data:
  	pools:   0 pools, 0 pgs
objects: 0 objects, 0 B
	usage:   0 B used, 0 B / 0 B avail
pgs:

Here we get a quick overview of what we have so far. Our cluster seems to be healthy and all three monitors are listed under services. Let’s go back to the admin and continue adding pieces.

$ ceph-deploy mgr create ip-10-0-0-124

For luminous+ builds a manager daemon is required. It’s responsible for monitoring the state of the Cluster and also manages modules/plugins.

Okay, now we have all the management in place, let’s add some storage to the cluster to make it actually useful, shall we?

First, we have to find out (on each target machine) the label of the drive we want to use. To fetch the list of available disks on a specific node, run

$ ceph-deploy disk list ip-10-0-0-104

Here’s a sample output:

ceph storage deploy sample output
$ ceph-deploy osd create --data /dev/nvme1n1 ip-10-0-0-124
$ ceph-deploy osd create --data /dev/nvme1n1 ip-10-0-0-216
$ ceph-deploy osd create --data /dev/nvme1n1 ip-10-0-0-104

In my case the label was nvme1n1 on all 3 machines (courtesy of AWS), so to add OSDs to the cluster I just ran these 3 commands.

At this point, our cluster is basically ready. We can run ceph status to see that our monitors, managers and OSDs are up and running. But nobody wants to SSH into a machine every time to check the status of the cluster. Luckily there’s a pretty neat dashboard that comes with Ceph, we just have to enable it.

…Or at least that’s what I thought. The dashboard was introduced in luminous release and was further improved in mimic. However, currently we’re deploying nautilus, the latest version of Ceph. After trying the usual way of enabling the dashboard via a manager

$ sudo ceph mgr module enable dashboard

we get an error message saying Error ENOENT: all mgr daemons do not support module 'dashboard', pass --force to force enablement.

Turns out, in nautilus the dashboard package is no longer installed by default. We can check the available modules by running

$ sudo ceph mgr module ls

and as expected, dashboard is not there, it comes in a form a separate package. So we have to install it first, luckily it’s pretty easy.

$ sudo apt install -y ceph-mgr-dashboard

Now we can enable it, right? Not so fast. There’s a dependency that has to be installed on all manager hosts, otherwise we get a slightly cryptic error message saying Error EIO: Module 'dashboard' has experienced an error and cannot handle commands: No module named routes.

$ sudo apt install -y python-routes

We’re all set to enable the dashboard module now. As it’s a public-facing page that requires login, we should set up a cert for SSL. For the sake of simplicity, I’ve just disabled the SSL feature. You should never do this in production, check out the official docs to see how to set up a cert properly. Also, we’ll need to create an admin user so we can log in to our dashboard.

$ sudo ceph mgr module enable dashboard
$ sudo ceph config set mgr mgr/dashboard/ssl false
$ sudo ceph dashboard ac-user-create admin secret administrator

By default, the dashboard is available on the host running the manager on port 8080. After logging in, we get an overview of the cluster status, and under the cluster menu, we get really detailed overviews of each running daemon.

ceph storage deployment dashboard
ceph cluster dashboard

If we try to navigate to the Filesystems or Object Gateway tabs, we get a notification that we haven’t configured the required resources to access these features. Our cluster can only be used as a block storage right now. We have to deploy a couple of extra things to extend its usability.

Quick detour: In case you’re looking for a company that can help you with Ceph, or DevOps in general, feel free to reach out to us at RisingStack!

Using the Ceph filesystem

Going back to our admin node, running

$ ceph-deploy mds create ip-10-0-0-124 ip-10-0-0-216 ip-10-0-0-104

will create metadata servers, that will be inactive for now, as we haven’t enabled the feature yet. First, we need to create two RADOS pools, one for the actual data and one for the metadata.

$ sudo ceph osd pool create cephfs_data 8
$ sudo ceph osd pool create cephfs_metadata 8

There are a couple of things to consider when creating pools that we won’t cover here. Please consult the documentation for further details.

After creating the required pools, we’re ready to enable the filesystem feature

$ sudo ceph fs new cephfs cephfs_metadata cephfs_data

The MDS daemons will now be able to enter an active state, and we are ready to mount the filesystem. We have two options to do that, via the kernel driver or as FUSE with ceph-fuse.

Before we continue with the mounting, let’s create a user keyring that we can use in both solutions for authorization and authentication as we have cephx enabled. There are multiple restrictions that can be set up when creating a new key specified in the docs. For example:

$ sudo ceph auth get-or-create client.user mon 'allow r' mds 'allow r, allow rw path=/home/cephfs' osd 'allow rw pool=cephfs_data' -o /etc/ceph/ceph.client.user.keyring

will create a new client key with the name user and output it into ceph.client.user.keyring. It will provide write access for the MDS only to the /home/cephfs directory, and the client will only have write access within the cephfs_data pool.

Mounting with the kernel

Now let’s create a dedicated directory and then use the key from the previously generated keyring to mount the filesystem with the kernel.

$ sudo mkdir /mnt/mycephfs
$ sudo mount -t ceph 13.53.114.94:6789:/ /mnt/mycephfs -o name=user,secret=AQBxnDFdS5atIxAAV0rL9klnSxwy6EFpR/EFbg==

Attaching with FUSE

Mounting the filesystem with FUSE is not much different either. It requires installing the ceph-fuse package.

$ sudo apt install -y ceph-fuse

Before we run the command we have to retrieve the ceph.conf and ceph.client.user.keyring files from the Ceph host and put the in /etc/ceph. The easiest solution is to use scp.

$ sudo scp ubuntu@13.53.114.94:/etc/ceph/ceph.conf /etc/ceph/ceph.conf
$ sudo scp ubuntu@13.53.114.94:/etc/ceph/ceph.client.user.keyring /etc/ceph/ceph.keyring

Now we are ready to mount the filesystem.

$ sudo mkdir cephfs
$ sudo ceph-fuse -m 13.53.114.94:6789 cephfs

Using the RADOS gateway

To enable the S3 management feature of the cluster, we have to add one final piece, the rados gateway.

$ ceph-deploy rgw create ip-10-0-0-124

For the dashboard, it’s required to create a radosgw-admin user with the system flag to enable the Object Storage management interface. We also have to provide the user’s access_key and secret_key to the dashboard before we can start using it.

$ sudo radosgw-admin user create --uid=rg_wadmin --display-name=rgw_admin --system
$ sudo ceph dashboard set-rgw-api-access-key <access_key>
$ sudo ceph dashboard set-rgw-api-secret-key <secret_key>

Using the Ceph Object Storage is really easy as RGW provides an interface identical to S3. You can use your existing S3 requests and code without any modifications, just have to change the connection string, access, and secret keys.

Ceph Storage Monitoring

The dashboard we’ve deployed shows a lot of useful information about our cluster, but monitoring is not its strongest suit. Luckily Ceph comes with a Prometheus module. After enabling it by running:

$ sudo ceph mgr module enable prometheus

A wide variety of metrics will be available on the given host on port 9283 by default. To make use of these exposed data, we’ll have to set up a prometheus instance.

I strongly suggest running the following containers on a separate machine from your Ceph cluster. In case you are just experimenting (like me) and don’t want to use a lot of VMs, make sure you have enough memory and CPU left on your virtual machine before firing up docker, as it can lead to strange behaviour and crashes if it runs out of resources.

There are multiple ways of firing up Prometheus, probably the most convenient is with docker. After installing docker on your machine, create a prometheus.yml file to provide the endpoint where it can access our Ceph metrics.

# /etc/prometheus.yml

scrape_configs:
  - job_name: 'ceph'
    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.
    static_configs:
    - targets: ['13.53.114.94:9283]

Then launch the container itself by running:

$ sudo docker run -p 9090:9090 -v /etc/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus

Prometheus will start scraping our data, and it will show up on its dashboard. We can access it on port 9090 on its host machine. Prometheus dashboard is great but does not provide a very eye-pleasing dashboard. That’s the main reason why it’s usually used in pair with Graphana, which provides awesome visualizations for the data provided by Prometheus. It can be launched with docker as well.

$ sudo docker run -d -p 3000:3000 grafana/grafana

Grafana is fantastic when it comes to visualizations, but setting up dashboards can be a daunting task. To make our lives easier, we can load one of the pre-prepared dashboards, for example this one.

ceph storage grafana monitoring

Ceph Deployment: Lessons Learned & Next Up

CEPH can be a great alternative to AWS S3 or other object storages when running in the public operating your service in the private cloud is simply not an option. The fact that it provides an S3 compatible interface makes it a lot easier to port other tools that were written with a “cloud first” mentality. It also plays nicely with Prometheus, thus you don’t need to worry about setting up proper monitoring for it, or you can swap it a more simple, more battle-hardened solution such as Nagios.

In this article, we deployed CEPH to bare virtual machines, but you might need to integrate it into your Kubernetes or Docker Swarm cluster. While it is perfectly fine to install it on VMs next to your container orchestration tool, you might want to leverage the services they provide when you deploy your CEPH cluster. If that is your use case, stay tuned for our next post covering CEPH where we’ll take a look at the black magic required to use CEPH on Docker Swarm and Kubernetes.

In the next CEPH tutorial which we’ll release next week, we’re going to take a look at valid ceph storage alternatives with Docker or with Kubernetes.

PS: Feel free to reach out to us at RisingStack in case you need help with Ceph or Ops in general!

Async Await in Node.js – How to Master it?

In this article, you will learn how you can simplify your callback or Promise based Node.js application with async functions (async await).

Whether you’ve looked at async/await and promises in JavaScript before, but haven’t quite mastered them yet, or just need a refresher, this article aims to help you.

async await nodejs explained

What are async functions in Node.js?

Async functions are available natively in Node and are denoted by the async keyword in their declaration. They always return a promise, even if you don’t explicitly write them to do so. Also, the await keyword is only available inside async functions at the moment – it cannot be used in the global scope.

In an async function, you can await any Promise or catch its rejection cause.

So if you had some logic implemented with promises:

function handler (req, res) {
  return request('https://user-handler-service')
    .catch((err) => {
      logger.error('Http error', err);
      error.logged = true;
      throw err;
    })
    .then((response) => Mongo.findOne({ user: response.body.user }))
    .catch((err) => {
      !error.logged && logger.error('Mongo error', err);
      error.logged = true;
      throw err;
    })
    .then((document) => executeLogic(req, res, document))
    .catch((err) => {
      !error.logged && console.error(err);
      res.status(500).send();
    });
}

You can make it look like synchronous code using async/await:

async function handler (req, res) {
  let response;
  try {
    response = await request('https://user-handler-service')  ;
  } catch (err) {
    logger.error('Http error', err);
    return res.status(500).send();
  }

  let document;
  try {
    document = await Mongo.findOne({ user: response.body.user });
  } catch (err) {
    logger.error('Mongo error', err);
    return res.status(500).send();
  }

  executeLogic(document, req, res);
}

Currently in Node you get a warning about unhandled promise rejections, so you don’t necessarily need to bother with creating a listener. However, it is recommended to crash your app in this case as when you don’t handle an error, your app is in an unknown state. This can be done either by using the --unhandled-rejections=strict CLI flag, or by implementing something like this:

process.on('unhandledRejection', (err) => { 
  console.error(err);
  process.exit(1);
})

Automatic process exit will be added in a future Node release – preparing your code ahead of time for this is not a lot of effort, but will mean that you don’t have to worry about it when you next wish to update versions.

Patterns with async functions in JavaScript

There are quite a couple of use cases when the ability to handle asynchronous operations as if they were synchronous comes very handy, as solving them with Promises or callbacks requires the use of complex patterns.

Since node@10.0.0, there is support for async iterators and the related for-await-of loop. These come in handy when the actual values we iterate over, and the end state of the iteration, are not known by the time the iterator method returns – mostly when working with streams. Aside from streams, there are not a lot of constructs that have the async iterator implemented natively, so we’ll cover them in another post.

Retry with exponential backoff

Implementing retry logic was pretty clumsy with Promises:

function request(url) {
  return new Promise((resolve, reject) => {
    setTimeout(() => {
      reject(`Network error when trying to reach ${url}`);
    }, 500);
  });
}

function requestWithRetry(url, retryCount, currentTries = 1) {
  return new Promise((resolve, reject) => {
    if (currentTries <= retryCount) {
      const timeout = (Math.pow(2, currentTries) - 1) * 100;
      request(url)
        .then(resolve)
        .catch((error) => {
          setTimeout(() => {
            console.log('Error: ', error);
            console.log(`Waiting ${timeout} ms`);
            requestWithRetry(url, retryCount, currentTries + 1);
          }, timeout);
        });
    } else {
      console.log('No retries left, giving up.');
      reject('No retries left, giving up.');
    }
  });
}

requestWithRetry('http://localhost:3000')
  .then((res) => {
    console.log(res)
  })
  .catch(err => {
    console.error(err)
  });

This would get the job done, but we can rewrite it with async/await and make it a lot more simple.

function wait (timeout) {
  return new Promise((resolve) => {
    setTimeout(() => {
      resolve()
    }, timeout);
  });
}

async function requestWithRetry (url) {
  const MAX_RETRIES = 10;
  for (let i = 0; i <= MAX_RETRIES; i++) {
    try {
      return await request(url);
    } catch (err) {
      const timeout = Math.pow(2, i);
      console.log('Waiting', timeout, 'ms');
      await wait(timeout);
      console.log('Retrying', err.message, i);
    }
  }
}

A lot more pleasing to the eye isn’t it?

Intermediate values

Not as hideous as the previous example, but if you have a case where 3 asynchronous functions depend on each other the following way, then you have to choose from several ugly solutions.

functionA returns a Promise, then functionB needs that value and functionC needs the resolved value of both functionA‘s and functionB‘s Promise.

Solution 1: The .then Christmas tree

function executeAsyncTask () {
  return functionA()
    .then((valueA) => {
      return functionB(valueA)
        .then((valueB) => {          
          return functionC(valueA, valueB)
        })
    })
}

With this solution, we get valueA from the surrounding closure of the 3rd then and valueB as the value the previous Promise resolves to. We cannot flatten out the Christmas tree as we would lose the closure and valueA would be unavailable for functionC.

Solution 2: Moving to a higher scope

function executeAsyncTask () {
  let valueA
  return functionA()
    .then((v) => {
      valueA = v
      return functionB(valueA)
    })
    .then((valueB) => {
      return functionC(valueA, valueB)
    })
}

In the Christmas tree, we used a higher scope to make valueA available as well. This case works similarly, but now we created the variable valueA outside the scope of the .then-s, so we can assign the value of the first resolved Promise to it.

This one definitely works, flattens the .then chain and is semantically correct. However, it also opens up ways for new bugs in case the variable name valueA is used elsewhere in the function. We also need to use two names — valueA and v — for the same value.

Are you looking for help with enterprise-grade Node.js Development?
Hire the Node developers of RisingStack!

Solution 3: The unnecessary array

function executeAsyncTask () {
  return functionA()
    .then(valueA => {
      return Promise.all([valueA, functionB(valueA)])
    })
    .then(([valueA, valueB]) => {
      return functionC(valueA, valueB)
    })
}

There is no other reason for valueA to be passed on in an array together with the Promise functionB then to be able to flatten the tree. They might be of completely different types, so there is a high probability of them not belonging to an array at all.

Solution 4: Write a helper function

const converge = (...promises) => (...args) => {
  let [head, ...tail] = promises
  if (tail.length) {
    return head(...args)
      .then((value) => converge(...tail)(...args.concat([value])))
  } else {
    return head(...args)
  }
}

functionA(2)
  .then((valueA) => converge(functionB, functionC)(valueA))

You can, of course, write a helper function to hide away the context juggling, but it is quite difficult to read, and may not be straightforward to understand for those who are not well versed in functional magic.

By using async/await our problems are magically gone:

async function executeAsyncTask () {
  const valueA = await functionA();
  const valueB = await functionB(valueA);
  return function3(valueA, valueB);
}

Multiple parallel requests with async/await

This is similar to the previous one. In case you want to execute several asynchronous tasks at once and then use their values at different places, you can do it easily with async/await:

async function executeParallelAsyncTasks () {
  const [ valueA, valueB, valueC ] = await Promise.all([ functionA(), functionB(), functionC() ]);
  doSomethingWith(valueA);
  doSomethingElseWith(valueB);
  doAnotherThingWith(valueC);
}

As we’ve seen in the previous example, we would either need to move these values into a higher scope or create a non-semantic array to pass these values on.

Array iteration methods

You can use mapfilter and reduce with async functions, although they behave pretty unintuitively. Try guessing what the following scripts will print to the console:

  1. map
function asyncThing (value) {
  return new Promise((resolve) => {
    setTimeout(() => resolve(value), 100);
  });
}

async function main () {
  return [1,2,3,4].map(async (value) => {
    const v = await asyncThing(value);
    return v * 2;
  });
}

main()
  .then(v => console.log(v))
  .catch(err => console.error(err));
  1. filter
function asyncThing (value) {
  return new Promise((resolve) => {
    setTimeout(() => resolve(value), 100);
  });
}

async function main () {
  return [1,2,3,4].filter(async (value) => {
    const v = await asyncThing(value);
    return v % 2 === 0;
  });
}

main()
  .then(v => console.log(v))
  .catch(err => console.error(err));
  1. reduce

function asyncThing (value) {
  return new Promise((resolve) => {
    setTimeout(() => resolve(value), 100);
  });
}

async function main () {
  return [1,2,3,4].reduce(async (acc, value) => {
    return await acc + await asyncThing(value);
  }, Promise.resolve(0));
}

main()
  .then(v => console.log(v))
  .catch(err => console.error(err));

Solutions:

  1. [ Promise { <pending> }, Promise { <pending> }, Promise { <pending> }, Promise { <pending> } ]
  2. [ 1, 2, 3, 4 ]
  3. 10

If you log the returned values of the iteratee with map you will see the array we expect: [ 2, 4, 6, 8 ]. The only problem is that each value is wrapped in a Promise by the AsyncFunction.

So if you want to get your values, you’ll need to unwrap them by passing the returned array to a Promise.all:

main()
  .then(v => Promise.all(v))
  .then(v => console.log(v))
  .catch(err => console.error(err));

Originally, you would first wait for all your promises to resolve and then map over the values:

function main () {
  return Promise.all([1,2,3,4].map((value) => asyncThing(value)));
}

main()
  .then(values => values.map((value) => value * 2))
  .then(v => console.log(v))
  .catch(err => console.error(err));

This seems a bit more simple, doesn’t it?

The async/await version can still be useful if you have some long running synchronous logic in your iteratee and another long-running async task.

This way you can start calculating as soon as you have the first value – you don’t have to wait for all the Promises to be resolved to run your computations. Even though the results will still be wrapped in Promises, those are resolved a lot faster then if you did it the sequential way.

What about filter? Something is clearly wrong…

Well, you guessed it: even though the returned values are [ false, true, false, true ], they will be wrapped in promises, which are truthy, so you’ll get back all the values from the original array. Unfortunately, all you can do to fix this is to resolve all the values and then filter them.

Reducing is pretty straightforward. Bear in mind though that you need to wrap the initial value into Promise.resolve, as the returned accumulator will be wrapped as well and has to be await-ed.

.. As it is pretty clearly intended to be used for imperative code styles.

To make your .then chains more “pure” looking, you can use Ramda’s pipeP and composeP functions.

Rewriting callback-based Node.js applications

Async functions return a Promise by default, so you can rewrite any callback based function to use Promises, then await their resolution. You can use the util.promisify function in Node.js to turn callback-based functions to return a Promise-based ones.

Rewriting Promise-based applications

Simple .then chains can be upgraded in a pretty straightforward way, so you can move to using async/await right away.

function asyncTask () {
  return functionA()
    .then((valueA) => functionB(valueA))
    .then((valueB) => functionC(valueB))
    .then((valueC) => functionD(valueC))
    .catch((err) => logger.error(err))
}
 

will turn into

async function asyncTask () {
  try {
    const valueA = await functionA();
    const valueB = await functionB(valueA);
    const valueC = await functionC(valueB);
    return await functionD(valueC);
  } catch (err) {
    logger.error(err);
  }
}

Rewriting Node.js apps with async await

  • If you liked the good old concepts of if-else conditionals and for/while loops,
  • if you believe that a try-catch block is the way errors are meant to be handled,

you will have a great time rewriting your services using async/await.

As we have seen, it can make several patterns a lot easier to code and read, so it is definitely more suitable in several cases than Promise.then() chains. However, if you are caught up in the functional programming craze of the past years, you might wanna pass on this language feature.

Are you already using async/await in production, or you plan on never touching it? Let’s discuss it in the comments below.

Are you looking for help with enterprise-grade Node.js Development?
Hire the Node developers of RisingStack!

Sometimes you do need Kubernetes! But how should you decide?

At RisingStack, we help companies to adopt cloud-native technologies, or if they have already done so, to get the most mileage out of them.

Recently, I’ve been invited to Google DevFest to deliver a presentation on our experiences working with Kubernetes.

Below I talk about an online learning and streaming platform where the decision to use Kubernetes has been contested both internally and externally since the beginning of its development.

The application and its underlying infrastructure were designed to meet the needs of the regulations of several countries:

  • The app should be able to run on-premises, so students’ data could never leave a given country. Also, the app had to be available as a SaaS product as well.
  • It can be deployed as a single-tenant system where a business customer only hosts one instance serving a handful of users, but some schools could have hundreds of users.
  • Or it can be deployed as a multi-tenant system where the client is e.g. a government and needs to serve thousands of schools and millions of users.

[elementor-template id="3483"]

The application itself was developed by multiple, geographically scattered teams, thus a Microservices architecture was justified, but both the distributed system and the underlying infrastructure seemed to be an overkill when we considered the fact that during the product’s initial entry, most of its customers needed small instances.

Was Kubernetes suited for the job, or was it an overkill? Did our client really need Kubernetes?

Let’s figure it out.

(Feel free to check out the video presentation, or the extended article version below!)

Let’s talk a bit about Kubernetes itself!

Kubernetes is an open-source container orchestration engine that has a vast ecosystem. If you run into any kind of problem, there’s probably a library somewhere on the internet that already solves it.

But Kubernetes also has a daunting learning curve, and initially, it’s pretty complex to manage. Cloud ops / infrastructure engineering is a complex and big topic in and of itself.

Kubernetes does not really mask away the complexity from you, but plunges you into deep water as it merely gives you a unified control plane to handle all those moving parts that you need to care about in the cloud.

So, if you’re just starting out right now, then it’s better to start with small things and not with the whole package straight away! First, deploy a VM in the cloud. Use some PaaS or FaaS solutions to play around with one of your apps. It will help you gradually build up the knowledge you need on the journey.

So you want to decide if Kubernetes is for you.

First and foremost, Kubernetes is for you if you work with containers! (It kinda speaks for itself for a container orchestration system). But you should also have more than one service or instance.

kubernetes-is-for-you-if

Kubernetes makes sense when you have a huge microservice architecture, or you have dedicated instances per tenant having a lot of tenants as well.

Also, your services should be stateless, and your state should be stored in databases outside of the cluster. Another selling point of Kubernetes is the fine gradient control over the network.

And, maybe the most common argument for using Kubernetes is that it provides easy scalability.

Okay, and now let’s take a look at the flip side of it.

Kubernetes is not for you if you don’t need scalability!

If your services rely heavily on disks, then you should think twice if you want to move to Kubernetes or not. Basically, one disk can only be attached to a single node, so all the services need to reside on that one node. Therefore you lose node auto-scaling, which is one of the biggest selling points of Kubernetes.

For similar reasons, you probably shouldn’t use k8s if you don’t host your infrastructure in the public cloud. When you run your app on-premises, you need to buy the hardware beforehand and you cannot just conjure machines out of thin air. So basically, you also lose node auto-scaling, unless you’re willing to go hybrid cloud and bleed over some of your excess load by spinning up some machines in the public cloud.

kubernetes-is-not-for-you-if

If you have a monolithic application that serves all your customers and you need some scaling here and there, then cloud service providers can handle it for you with autoscaling groups.

There is really no need to bring in Kubernetes for that.

Let’s see our Kubernetes case-study!

Maybe it’s a little bit more tangible if we talk about an actual use case, where we had to go through the decision making process.

online-learning-platform-kubernetes

Online Learning Platform is an application that you could imagine as if you took your classroom and moved it to the internet.

You can have conference calls. You can share files as handouts, you can have a whiteboard, and you can track the progress of your students.

This project started during the first wave of the lockdowns around March, so one thing that we needed to keep in mind is that time to market was essential.

In other words: we had to do everything very, very quickly!

This product targets mostly schools around Europe, but it is now used by corporations as well.

So, we’re talking about millions of users from the point we go to the market.

The product needed to run on-premise, because one of the main targets were governments.

Initially, we were provided with a proposed infrastructure where each school would have its own VM, and all the services and all the databases would reside in those VMs.

Handling that many virtual machines, properly handling rollouts to those, and monitoring all of them sounded like a nightmare to begin with. Especially if we consider the fact that we only had a couple of weeks to go live.

After studying the requirements and the proposal, it was time to call the client to..

Discuss the proposed infrastructure.

So the conversation was something like this:

  • “Hi guys, we would prefer to go with Kubernetes because to handle stuff at that scale, we would need a unified control plane that Kubernetes gives us.”
  • "Yeah, sure, go for it."

And we were happy, but we still had a couple of questions:

  • “Could we, by any chance, host it on the public cloud?”
  • "Well, no, unfortunately. We are negotiating with European local governments and they tend to be squeamish about sending their data to the US. "

Okay, anyways, we can figure something out…

  • “But do the services need filesystem access?”
  • "Yes, they do."

Okay, crap! But we still needed to talk to the developers so all was not lost.

Let’s call the developers!

It turned out that what we were dealing with was an usual microservice-based architecture, which consisted of a lot of services talking over HTTP and messaging queues.

Each service had its own database, and most of them stored some files in Minio.

kubernetes-app-architecture

In case you don’t know it, Minio is an object storage system that implements the S3 API.

Now that we knew the fine-grained architectural layout, we gathered a few more questions:

  • “Okay guys, can we move all the files to Minio?”
  • "Yeah, sure, easy peasy."

So, we were happy again, but there was still another problem, so we had to call the hosting providers:

  • “Hi guys, do you provide hosted Kubernetes?”
  • "Oh well, at this scale, we can manage to do that!"

So, we were happy again, but..

Just to make sure, we wanted to run the numbers!

Our target was to be able to run 60 000 schools on the platform in the beginning, so we had to see if our plans lined up with our limitations!

We shouldn’t have more than 150 000 total pods!

10 (pod/tenant) times 6000 tenants is 60 000 Pods. We’re good!

We shouldn’t have more than 300 000 total containers!

It’s one container per pod, so we’re still good.

We shouldn’t have more than 100 pods per node and no more than 5 000 nodes.

Well, what we have is 60 000 pods over 100 pod per node. That’s already 6 000 nodes, and that’s just the initial rollout, so we’re already over our 5 000 nodes limit.

kubernetes-limitations

Okay, well… Crap!

But, is there a solution to this?

Sure, it’s federation!

We could federate our Kubernetes clusters..

..and overcome these limitations.

We have worked with federated systems before, so Kubernetes surely provides something for that, riiight? Well yeah, it does… kind of.

It’s the stable Federation v1 API, which is sadly deprecated.

kubernetes-federation-v1

Then we saw that Kubernetes Federation v2 is on the way!

It was still in alpha at the time when we were dealing with this issue, but the GitHub page said it was rapidly moving towards beta release. By taking a look at the releases page we realized that it had been overdue by half a year by then.

Since we only had a short period of time to pull this off, we really didn’t want to live that much on the edge.

So what could we do? We could federate by hand! But what does that mean?

In other words: what could have been gained by using KubeFed?

Having a lot of services would have meant that we needed a federated Prometheus and Logging (be it Graylog or ELK) anyway. So the two remaining aspects of the system were rollout / tenant generation, and manual intervention.

Manual intervention is tricky. To make it easy, you need a unified control plane where you can eyeball and modify anything. We could have built a custom one that gathers all information from the clusters and proxies all requests to each of them. However, that would have meant a lot of work, which we just did not have the time for. And even if we had the time to do it, we would have needed to conduct a cost/benefit analysis on it.

The main factor in the decision if you need a unified control plane for everything is scale, or in other words, the number of different control planes to handle.

The original approach would have meant 6000 different planes. That’s just way too much to handle for a small team. But if we could bring it down to 20 or so, that could be bearable. In that case, all we need is an easy mind map that leads from services to their underlying clusters. The actual route would be something like:

Service -> Tenant (K8s Namespace) -> Cluster.

The Service -> Namespace mapping is provided by Kubernetes, so we needed to figure out the Namespace -> Cluster mapping.

This mapping is also necessary to reduce the cognitive overhead and time of digging around when an outage may happen, so it needs to be easy to remember, while having to provide a more or less uniform distribution of tenants across Clusters. The most straightforward way seemed to be to base it on Geography. I’m the most familiar with Poland’s and Hungary’s Geography, so let’s take them as an example.

Poland comprises 16 voivodeships, while Hungary comprises 19 counties as main administrative divisions. Each country’s capital stands out in population, so they have enough schools to get a cluster on their own. Thus it only makes sense to create clusters for each division plus the capital. That gives us 17 or 20 clusters.

So if we get back to our original 60 000 pods, and 100 pod / tenant limitation, we can see that 2 clusters are enough to host them all, but that leaves us no room for either scaling or later expansions. If we spread them across 17 clusters – in the case of Poland for example – that means we have around 3.500 pods / cluster and 350 nodes, which is still manageable.

This could be done in a similar fashion for any European country, but still needs some architecting when setting up the actual infrastructure. And when KubeFed becomes available (and somewhat battle tested) we can easily join these clusters into one single federated cluster.

Great, we have solved the problem of control planes for manual intervention. The only thing left was handling rollouts..

gitops-kubernetes-federation

As I mentioned before, several developer teams had been working on the services themselves, and each of them already had their own Gitlab repos and CIs. They already built their own Docker images, so we simply needed a place to gather them all, and roll them out to Kubernetes. So we created a GitOps repo where we stored the helm charts and set up a GitLab CI to build the actual releases, then deploy them.

From here on, it takes a simple loop over the clusters to update the services when necessary.

The other thing we needed to solve was tenant generation.

It was easy as well, because we just needed to create a CLI tool which could be set up by providing the school’s name, and its county or state.

tenant-generation-kubernetes

That’s going to designate its target cluster, and then push it to our Gitops repo, and that basically triggers the same rollout as new versions.

We were almost good to go, but there was still one problem: on-premises.

Although our hosting providers turned into some kind of public cloud (or something we can think of as public clouds), we were also targeting companies who want to educate their employees.

Huge corporations – like a Bank – are just as squeamish about sending their data out to the public internet as governments, if not more..

So we needed to figure out a way to host this on servers within vaults completely separated from the public internet.

kubespray

In this case, we had two main modes of operation.

  • One is when a company just wanted a boxed product and they didn’t really care about scaling it.
  • And the other one was where they expected it to be scaled, but they were prepared to handle this.

In the second case, it was kind of a bring your own database scenario, so you could set up the system in a way that we were going to connect to your database.

And in the other case, what we could do is to package everything — including databases — in one VM, in one Kubernetes cluster. But! I just wrote above that you probably shouldn’t use disks and shouldn’t have databases within your cluster, right?

However, in that case, we already had a working infrastructure.

Kubernetes provided us with infrastructure as code already, so it only made sense to use that as a packaging tool as well, and use Kubespray to just spray it to our target servers.

It wasn’t a problem to have disks and DBs within our cluster because the target were companies that didn’t want to scale it anyway.

So it’s not about scaling. It is mostly about packaging!

Previously I told you, that you probably don’t want to do this on-premises, and this is still right! If that’s your main target, then you probably shouldn’t go with Kubernetes.

However, as our main target was somewhat of a public cloud, it wouldn’t have made sense to just recreate the whole thing – basically create a new product in a sense – for these kinds of servers.

So as it is kind of a spin-off, it made sense here as well as a packaging solution.

Basically, I’ve just given you a bullet point list to help you determine whether Kubernetes is for you or not, and then I just tore it apart and threw it into a basket.

And the reason for this is – as I also mentioned:

Cloud ops is difficult!

There aren’t really one-size-fits-all solutions, so basing your decision on checklists you see on the internet is definitely not a good idea.

We’ve seen that a lot of times where companies adopt Kubernetes because it seems to fit, but when they actually start working with it, it turns out to be an overkill.

If you want to save yourself about a year or two of headache, it’s a lot better to first ask an expert, and just spend a couple of hours or days going through your use cases, discussing those and save yourself that year of headache.

In case you’re thinking about adopting Kubernetes, or getting the most out of it, don’t hesitate to reach out to us at info@risingstack.com, or by using the contact form below!

Distributed Load Testing with Jmeter

Many of you have probably used apache Jmeter for load testing before. Still, it is easy to run into the limits imposed by running it on just one machine when trying to make sure that our API will be able to serve hundreds of thousands or even millions of users.

We can get around this issue by deploying and running our tests to multiple machines in the cloud.

In this article, we will take a look at one way to distribute and run Jmeter tests along multiple droplets on DigitalOcean using Terraform, Ansible, and a little bit of bash scripting to automate the process as much as possible.

Background: During the COVID19 outbreak induced lockdowns, we’ve been tasked by a company (who builds an e-learning platform primarily for schools) to build out an infrastructure that is:

  • geo redundant,
  • supports both single and multi tenant deployments ,
  • can be easily scaled to serve at least 1.5 million users in huge bursts,
  • and runs on-premises.

To make sure the application is able to handle these requirements, we needed to set up the infrastructure, and model a reasonably high burst in requests to get an idea about the load the application and its underlying infrastructure is able to serve.

In this article, we’ll share practical advice and some of the scripts we used to automate the load-testing process using Jmeter, Terraform and Ansible.

Let’s Start!

Have these tools installed before you begin!

brew install ansible
brew install terraform
brew install jmeter

You can just run them from your own machine. The full codebase is available on Github at RisingStack/distributed-loadtests-jmeter for your convenience.

Why do we use Jmeter for distributed load testing?

Jmeter is not my favorite tool for load testing owing mostly to the fact that scripting it is just awkward. But looking at the other tools that support distribution, it seems to be the best free one for now. K6 looks good, but right now it does not support distribution outside the paid, hosted version. Locust is another interesting one, but it’s focusing too much on random test picking, and if that’s not what I’m looking for, it is quite awkward to use as well – just not flexible enough right now.

So, back to Jmeter!

Terraform is infrastructure as code, which allows us to describe the resources we want to use in our deployment and configure the droplets so we have them ready for running some tests. This will, in turn, be deployed by Ansible to our cloud service provider of choice, DigitalOcean – though with some changes, you can make this work with any other provider, as well as your on-premise machines if you wish so.

Deploying the infrastructure

There will be two kinds of instances we’ll use:

  • primary, of which we’ll have one coordinating the testing,
  • and runners, that we can have any number of.

In the example, we’re going to go with two, but we’ll see that it is easy to change this when needed.

You can check the variables.tf file to see what we’ll use. You can use these to customise most aspects of the deployment to fit your needs. This file holds the vars that will be plugged into the other template files – main.tf and provider.tf.

The one variable you’ll need to provide to Terraform for the example setup to work is your DigitalOcean api token, that you can export like this from the terminal:

export TF_VAR_do_token=DO_TOKEN

Should you wish to change the number of test runner instances, you can do so by exporting this other environment variable:

export TF_VAR_instance_count=2

You will need to generate two ssh key pairs, one for the root user, and one for a non-privileged user. These will be used by Ansible, which uses ssh to deploy the testing infrastructure as it is agent-less. We will also use the non-privileged user when starting the tests for copying over files and executing commands on the primary node. The keys should be set up with correct permissions, otherwise, you’ll just get an error.

Set the permissions to 600 or 700 like this:

chmod 600 /path/to/folder/with/keys/*

To begin, we should open a terminal in the terraform folder, and call terraform init which will prepare the working directory. Thisl needs to be called again if the configuration changes.

You can use terraform plan that will output a summary of what the current changes will look like to the console to double-check if everything is right. At the first run, it will be what the deployment will look like.

Next, we call terraform apply which will actually apply the changes according to our configuration, meaning we’ll have our deployment ready when it finishes! It also generates a .tfstate file with all the information about said deployment.

If you wish to dismantle the deployment after the tests are done, you can use terraform destroy. You’ll need the .tfstate file for this to work though! Without the state file, you need to delete the created droplets by hand, and also remove the ssh key that has been added to DigitalOcean.

Running the Jmeter tests

The shell script we are going to use for running the tests is for convenience – it consists of copying the test file to our primary node, cleaning up files from previous runs, running the tests, and then fetching the results.

#!/bin/bash

set -e

# Argument parsing, with options for long and short names
for i in "$@"
do
case $i in
    -o=*|--out-file=*)
    # i#*= This removes the shortest substring ending with
    # '=' from the value of variable i - leaving us with just the
    # value of the argument (i is argument=value)
    OUTDIR="${i#*=}"
    shift
    ;;
    -f=*|--test-file=*)
    TESTFILE="${i#*=}"
    shift
    ;;
    -i=*|--identity-file=*)
    IDENTITYFILE="${i#*=}"
    shift
    ;;
    -p=*|--primary-ip=*)
    PRIMARY="${i#*=}"
    shift
    ;;
esac
done

# Check if we got all the arguments we'll need
if [ -z "$TESTFILE" ] || [ ! -f "$TESTFILE" ]; then
    echo "Please provide a test file"
    exit 1
fi

if [ -z "$OUTDIR" ]; then
    echo "Please provide a result destination directory"
    exit 1
fi

if [ -z "$IDENTITYFILE" ]; then
    echo "Please provide an identity file for ssh access"
    exit 1
fi

if [ -z "$PRIMARY" ]; then
  PRIMARY=$(terraform output primary_address)
fi

# Copy the test file to the primary node
scp -i "$IDENTITYFILE" -o IdentitiesOnly=yes -oStrictHostKeyChecking=no "$TESTFILE" "runner@$PRIMARY:/home/runner/jmeter/test.jmx"
# Remove files from previous runs if any, then run the current test
ssh -i "$IDENTITYFILE" -o IdentitiesOnly=yes -oStrictHostKeyChecking=no "runner@$PRIMARY" << "EOF"
 rm -rf /home/runner/jmeter/result
 rm -f /home/runner/jmeter/result.log
 cd jmeter/bin ; ./jmeter -n -r -t ../test.jmx -l ../result.log -e -o ../result -Djava.rmi.server.hostname=$(hostname -I | awk ' {print $1}')
EOF
# Get the results
scp -r -i "$IDENTITYFILE" -o IdentitiesOnly=yes -oStrictHostKeyChecking=no "runner@$PRIMARY":/home/runner/jmeter/result "$OUTDIR"

Running the script will require the path to the non-root ssh key. The call will look something like this:

bash run.sh -i=/path/to/non-root/ssh/key  -f=/path/to/test/file -o=/path/to/results/dir

You can also supply the IP of the primary node using -p= or --primary-ip= in case you don’t have access to the .tfstate file. Otherwise, the script will ask terraform for the IP.

Jmeter will then take care of distributing the tests across the runner nodes, and it will aggregate the data when they finish. The only thing we need to keep in mind is that the number of users we set for our test to use will not be split but will be multiplied. As an example, if you set the user count to 100, each runner node will then run the tests with 100 users.

And that’s how you can use Terraform and Ansible to run your distributed Jmeter tests on DigitalOcean!

Check this page for more on string manipulation in bash.

Looking for DevOps & Infra Experts?

In case you’re looking for expertise in infrastructure related matters, I’d recommend to read our articles and ebooks on the topic, and to check out our various service pages:

An early draft of this article was written by Mate Boer, and then subsequently rewritten by Janos Kubisch – both engineers at RisingStack.

Node.js Async Best Practices & Avoiding the Callback Hell

In this post, we cover what tools and techniques you have at your disposal when handling Node.js asynchronous operations: async.jspromises, and async functions.

After reading this article, you’ll know how to use the latest async tools at your disposal provided by Node.js!

Node.js at Scale is a collection of articles focusing on the needs of companies with bigger Node.js installations and advanced Node developers. Chapters:

See all chapters of Node.js at Scale:

Asynchronous programming in Node.js

Previously we have gathered a strong knowledge about asynchronous programming in JavaScript and understood how the Node.js event loop works.

If you have not read these articles, I highly recommend them as introductions!

The Problem with Node.js Async

Node.js itself is single-threaded, but some tasks can run in parallel thanks to its asynchronous nature.

But what does running in parallel mean in practice?

Since we program a single-threaded VM, it is essential that we do not block execution by waiting for I/O, but handle operations concurrently with the help of Node.js’s event-driven APIs.

Let’s take a look at some fundamental patterns, and learn how we can write resource-efficient, non-blocking code, with the built-in solutions of Node.js.

The Classical Approach – Callbacks

Let’s take a look at these simple async operations. They do nothing special, just fire a timer and call a function once the timer finished.

function fastFunction (done) {
  setTimeout(function () {
    done()
  }, 100)
}

function slowFunction (done) {
  setTimeout(function () {
    done()
  }, 300)
}

Seems easy, right?

Our higher-order functions can be executed sequentially or in parallel with the basic “pattern” by nesting callbacks – but using this method can lead to an untameable callback-hell.

function runSequentially (callback) {
  fastFunction((err, data) => {
    if (err) return callback(err)
    console.log(data)   // results of a
  
    slowFunction((err, data) => {
      if (err) return callback(err)
      console.log(data) // results of b
  
      // here you can continue running more tasks
    })
  })
}

Never use the nested callback approach for handling asynchronous Node,js operations!

Avoiding Callback Hell with Control Flow Managers

To become an efficient Node.js developer, you have to avoid the constantly growing indentation level, produce clean and readable code and be able to handle complex flows.

Let me show you some of the tools we can use to organize our code in a nice and maintainable way!

#1: Using Promises

There have been native promises in javascript since 2014, receiving an important boost in performance in Node.js 8. We will make use of them in our functions to make them non-blocking – without the traditional callbacks. The following example will call the modified version of both our previous functions in such a manner:

function fastFunction () {
  return new Promise((resolve, reject) => {
    setTimeout(function () {
      console.log('Fast function done')
      resolve()
    }, 100)
  })
}

function slowFunction () {
  return new Promise((resolve, reject) => {
    setTimeout(function () {
      console.log('Slow function done')
      resolve()
    }, 300)
  })
}

function asyncRunner () {
    return Promise.all([slowFunction(), fastFunction()])
}

Please note that Promise.all will fail as soon as any of the promises inside it fails.

The previous functions have been modified slightly to return promises. Our new function, asyncRunner, will also return a promise, that will resolve when all the contained functions resolve, and this also means that wherever we call our asyncRunner, we’ll be able to use the .then and .catch methods to deal with the possible outcomes:

asyncRunner()
  .then(([ slowResult, fastResult ]) => {
    console.log('All operations resolved successfully')
  })
  .catch((error) => {
    console.error('There has been an error:', error)
  })

Since node@12.9.0, there is a method called promise.allSettled, that we can use to get the result of all the passed in promises regardless of rejections. Much like Promise.all, this function expects an array of promises, and returns an array of objects that has a status of “fulfilled” or “rejected”, and either the resolved value or the error that occurred.

function failingFunction() {
  return new Promise((resolve, reject) => {
    reject(new Error('This operation will surely fail!'))
  })
}

function asyncMixedRunner () {
    return Promise.allSettled([slowFunction(), failingFunction()])
}

asyncMixedRunner()
    .then(([slowResult, failedResult]) => {
        console.log(slowResult, failedResult)
    })

In previous node versions, where .allSettled is not available, we can implement our own version in just a few lines:

function homebrewAllSettled(promises) {
  return Promise.all(promises.map((promise) => {
    return promise
      .then((value) => {
        return { status: 'fulfilled', value }
      })
      .catch((error) => {
        return { status: 'rejected', error }
      })
  }))
}

Serial task execution

To make sure your tasks run in a specific order – maybe successive functions need the return value of previous ones, or depend on the run of previous functions less directly – which is basically the same as _.flow for functions that return a Promise. As long as it’s missing from everyone’s favorite utility library, you can easily create a chain from an array of your async functions:

function serial(asyncFunctions) {
    return asyncFunctions.reduce(function(functionChain, nextFunction) {
        return functionChain.then(
            (previousResult) => nextFunction(previousResult)
        );
    }, Promise.resolve());
}

serial([parameterValidation, dbQuery, serviceCall ])
   .then((result) => console.log(`Operation result: ${result}`))
   .catch((error) => console.log(`There has been an error: ${error}`))

In case of a failure, this will skip all the remaining promises, and go straight to the error handling branch. You can tweak it some more in case you need the result of all of the promises regardless if they resolved or rejected.

function serial(asyncFunctions) {
    return asyncFunctions.map(function(functionChain, nextFunction) {
        return functionChain
            .then((previousResult) => nextFunction(previousResult))
            .then(result => ({ status: 'fulfilled', result }))
            .catch(error => ({ status: 'rejected', error }));
    }, Promise.resolve());
}

Converting callback functions to promises

Node also provides a handy utility function called “promisify”, that you can use to convert any old function expecting a callback that you just have to use into one that returns a promise. All you need to do is import it in your project:

const promisify = require('util').promisify;
function slowCallbackFunction (done) {
  setTimeout(function () {
    done()
  }, 300)
}
const slowPromise = promisify(slowCallbackFunction);

slowPromise()
  .then(() => {
    console.log('Slow function resolved')
  })
  .catch((error) => {
    console.error('There has been an error:', error)
  })

It’s actually not that hard to implement a promisify function of our own, to learn more about how it works. We can even handle additional arguments that our wrapped functions might need!

function homebrewPromisify(originalFunction, originalArgs = []) {
  return new Promise((resolve, reject) => {
    originalFunction(...originalArgs, (error, result) => {
      if (error) return reject(error)
      return resolve(result)
    })
  })
}

We just wrap the original callback-based function in a promise, and then reject or resolve based on the result of the operation.

Easy as that!

For better support of callback based code – legacy code, ~50% of the npm modules – Node also includes a callbackify function, essentially the opposite of promisify, which takes an async function that returns a promise, and returns a function that expects a callback as its single argument.

const callbackify = require('util').callbackify
const callbackSlow = callbackify(slowFunction)

callbackSlow((error, result) => {
  if (error) return console.log('Callback function received an error')
  return console.log('Callback resolved without errors')
})

#2: Meet Async – aka how to write async code in 2020

We can use another javascript feature since node@7.6 to achieve the same thing: the async and await keywords. They allow you to structure your code in a way that is almost synchronous looking, saving us the .then chaining as well as callbacks:

const promisify = require('util').promisify;

async function asyncRunner () {
    try {
      const slowResult = await promisify(slowFunction)()
      const fastResult = await promisify(fastFunction)()
      console.log('all done')
      return [
        slowResult,
        fastResult
      ]
    } catch (error) {
      console.error(error)
    }
}

This is the same async runner we’ve created before, but it does not require us to wrap our code in .then calls to gain access to the results. For handling errors, we have the option to use try & catch blocks, as presented above, or use the same .catch calls that we’ve seen previously with promises. This is possible because async-await is an abstraction on top of promises – async functions always return a promise, even if you don’t explicitly declare them to do so.

The await keyword can only be used inside functions that have the async tag. This also means that we cannot currently utilize it in the global scope.

Since Node 10, we also have access to the promise.finally method, which allows us to run code regardless of whether the promise resolve or rejected. It can be used to run tasks that we had to call in both the .then and .catch paths previously, saving us some code duplication.

Using all of this in Practice

As we have just learned several tools and tricks to handle async, it is time to do some practice with fundamental control flows to make our code more efficient and clean.

Let’s take an example and write a route handler for our web app, where the request can be resolved after 3 steps: validateParamsdbQuery and serviceCall.

If you’d like to write them without any helper, you’d most probably end up with something like this. Not so nice, right?

// validateParams, dbQuery, serviceCall are higher-order functions
// DONT
function handler (done) {
  validateParams((err) => {
    if (err) return done(err)
    dbQuery((err, dbResults) => {
      if (err) return done(err)
      serviceCall((err, serviceResults) => {
        done(err, { dbResults, serviceResults })
      })
    })
  })
}

Instead of the callback-hell, we can use promises to refactor our code, as we have already learned:

// validateParams, dbQuery, serviceCall are higher-order functions
function handler () {
  return validateParams()
    .then(dbQuery)
    .then(serviceCall)
    .then((result) => {
      console.log(result)
      return result
    })
    .catch(console.log.bind(console))
}

Let’s take it a step further! Rewrite it to use the async and await keywords:

// validateParams, dbQuery, serviceCall are thunks
async function handler () {
  try {
    await validateParams()
    const dbResults = await dbQuery()
    const serviceResults = await serviceCall()
    return { dbResults, serviceResults }
  } catch (error) {
    console.log(error)
  }
}

It feels like a “synchronous” code but still doing async operations one after each other.

Essentially, a new callback is injected into the functions, and this is how async knows when a function is finished.

Takeaway rules for Node.js & Async

Fortunately, Node.js eliminates the complexities of writing thread-safe code. You just have to stick to these rules to keep things smooth:

As a rule of thumb, prefer async, because using a non-blocking approach gives superior performance over the synchronous scenario, and the async – await keywords gives you more flexibility in structuring your code. Luckily, most libraries now have promise based APIs, so compatibility is rarely an issue, and can be solved with util.promisify should the need arise.

If you have any questions or suggestions for the article, please let me know in the comments!

In case you’re looking for help with Node.js consulting or development, feel free to reach out to us! Our team of experienced engineers is ready to speed up your development process, or educate your team on JavaScript, Node, React, Microservices and Kubernetes.

In the next part of the Node.js at Scale series, we take a look at Event Sourcing with Examples.

This article was originally written by Tamas Hodi, and was released on 2017, January 17. The revised second edition was authored by Janos Kubisch and Tamas Hodi and it was released on 2020 February 10.