OTP and Building Real Applications - Learn Elixir Part 4

In this part, we will look at OTP (Open Telecom Platform), which is an important part of Elixir. OTP helps you build applications that can handle many users at the same time, recover from errors, and run on more than one machine.

What is OTP?

OTP is a complete framework with tools and patterns for building reliable applications. It was first made by Ericsson for telecom systems, but now anyone using Elixir can use it.

The main idea behind OTP is that you cannot stop all failures, but you can make your system recover from them. If something goes wrong, OTP helps your application notice the problem and fix itself.

The OTP Philosophy

OTP is based on a few main ideas:

1. Process Isolation: Each part of your program runs in its own process.
2. Supervision: Processes are watched and restarted if they fail.
3. Message Passing: Processes talk to each other by sending messages, not by sharing memory.
4. Fault Tolerance: Problems are kept in one place and do not crash the whole system.

OTP gives you tools to use these ideas without too much extra work.

GenServer: The Basic Building Block

GenServer is the OTP feature you will use most often. It gives you a standard way to create processes that keep state and answer messages. You can think of it as a pattern for building services that need to remember things.

Why use GenServer?

Without GenServer, you would need to write all the code for message passing, state, and error handling yourself. GenServer gives you a simple interface and takes care of the hard parts.

Here's what a basic GenServer looks like:

defmodule Counter do
  use GenServer

  # Client API - what other processes call
  def start_link(initial_value \\ 0) do
    GenServer.start_link(__MODULE__, initial_value, name: __MODULE__)
  end

  def increment do
    GenServer.call(__MODULE__, :increment)
  end

  def get_count do
    GenServer.call(__MODULE__, :get_count)
  end

  # Server callbacks - what the GenServer does internally
  @impl true
  def init(initial_value) do
    {:ok, initial_value}
  end

  @impl true
  def handle_call(:increment, _from, state) do
    new_state = state + 1
    {:reply, new_state, new_state}
  end

  @impl true
  def handle_call(:get_count, _from, state) do
    {:reply, state, state}
  end
end

The key insight is that GenServer separates the client interface (what other processes call) from the server implementation (how the state is managed). The @impl true annotations tell the compiler that these are GenServer callbacks.

Understanding the Callbacks

Let's break down what each callback does:

• init/1: Called when the GenServer starts. Returns {:ok, initial_state}
• handle_call/3: Handles synchronous requests (calls). Returns {:reply, response, new_state}
• handle_cast/2: Handles asynchronous requests (casts). Returns {:noreply, new_state}
• handle_info/2: Handles other messages (like from send/2). Returns {:noreply, new_state}

The state flows through these callbacks, and each one can transform it as needed.

When to Use Call vs Cast

• GenServer.call/2: Synchronous - the caller waits for a response
• GenServer.cast/2: Asynchronous - the caller doesn't wait

Use call when you need a response, cast when you just want to send a message and continue.

Supervisors: The Safety Net

Supervisors are what make Elixir applications fault-tolerant. They monitor child processes and restart them when they fail. Think of them as the parent process that watches over their children.

Why Supervision Matters

In traditional applications, if a component fails, it might bring down the entire system. With supervisors, failures are isolated and handled automatically. If a worker process crashes, the supervisor restarts it. If it keeps crashing, the supervisor can try different strategies.

Basic Supervisor

defmodule MyApp.Supervisor do
  use Supervisor

  def start_link(init_arg) do
    Supervisor.start_link(__MODULE__, init_arg, name: __MODULE__)
  end

  @impl true
  def init(_init_arg) do
    children = [
      # Start the counter
      {Counter, 0},
      
      # Start a worker with specific options
      {ChatRoom, "general"}
    ]

    Supervisor.init(children, strategy: :one_for_one)
  end
end

The supervisor defines a list of child processes to start and monitor. Each child is specified as a tuple {module, arguments}.

Supervisor Strategies

Supervisors have different strategies for handling failures:

• :one_for_one: Restart only the failed child
• :one_for_all: Restart all children if any child fails
• :rest_for_one: Restart the failed child and all children started after it
• :simple_one_for_one: Restart only the failed child (for dynamic children)

Choose the strategy based on how your processes depend on each other. If they're independent, use :one_for_one. If they need to be consistent, use :one_for_all.

Building Web Applications with Phoenix

Phoenix is Elixir's web framework, and it's designed for high-performance real-time applications, built on the same principles as OTP.

Why Phoenix?

Phoenix is fast and it can handle thousands of concurrent connections with minimal resource usage. It's also great for real-time features like chat, live updates, and collaborative editing.

Creating a Phoenix Application

# Install Phoenix
mix archive.install hex phx_new

# Create a new Phoenix application
mix phx.new my_app --no-ecto

# Create with database support
mix phx.new my_app --database postgres

The --no-ecto flag creates a Phoenix app without database support, which is good for learning. The --database postgres flag sets up PostgreSQL integration.

Understanding Phoenix Structure

A Phoenix application has a clear structure:

my_app/
├── lib/
│   ├── my_app/          # Your business logic
│   │   ├── application.ex
│   │   └── ...
│   └── my_app_web/      # Web-specific code
│       ├── endpoint.ex
│       ├── router.ex
│       ├── controllers/
│       └── templates/
├── config/              # Configuration files
├── test/                # Tests
└── mix.exs

The separation between my_app and my_app_web is important - it keeps your business logic separate from your web interface.

Routing in Phoenix

Phoenix routing is powerful and flexible:

defmodule MyAppWeb.Router do
  use MyAppWeb, :router

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_live_flash
    plug :put_root_layout, html: {MyAppWeb.Layouts, :root}
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/", MyAppWeb do
    pipe_through :browser

    get "/", PageController, :home
    resources "/users", UserController
  end

  scope "/api", MyAppWeb do
    pipe_through :api

    resources "/users", UserController, only: [:index, :show, :create]
  end
end

Pipelines are groups of plugs (middleware) that process requests. The :browser pipeline handles HTML requests, while the :api pipeline handles JSON requests.

Controllers

Controllers handle HTTP requests and return responses:

defmodule MyAppWeb.UserController do
  use MyAppWeb, :controller

  def index(conn, _params) do
    users = [
      %{id: 1, name: "John", email: "john@example.com"},
      %{id: 2, name: "Jane", email: "jane@example.com"}
    ]
    
    render(conn, :index, users: users)
  end

  def show(conn, %{"id" => id}) do
    user = %{id: id, name: "John", email: "john@example.com"}
    render(conn, :show, user: user)
  end
end

The conn parameter is the connection struct that contains all the request and response data. The params parameter contains the request parameters.

LiveView for Real-time Features

LiveView is one of Phoenix's most powerful features. It enables real-time, server-rendered HTML without writing JavaScript:

defmodule MyAppWeb.ChatLive do
  use MyAppWeb, :live_view

  def mount(_params, _session, socket) do
    if connected?(socket) do
      Phoenix.PubSub.subscribe(MyApp.PubSub, "chat:general")
    end

    {:ok, assign(socket, messages: [], current_user: "Anonymous")}
  end

  def handle_event("send_message", %{"message" => message}, socket) do
    if String.trim(message) != "" do
      Phoenix.PubSub.broadcast(
        MyApp.PubSub,
        "chat:general",
        {:new_message, socket.assigns.current_user, message}
      )
    end

    {:noreply, socket}
  end

  def handle_info({:new_message, user, message}, socket) do
    new_message = %{
      id: :rand.uniform(1000),
      user: user,
      message: message,
      timestamp: DateTime.utc_now()
    }

    {:noreply, update(socket, :messages, fn messages -> [new_message | messages] end)}
  end
end

LiveView maintains a persistent connection between the browser and server. When the server state changes, it automatically updates the browser. This makes real-time features much easier to implement.

Working with Databases using Ecto

Ecto is Elixir's database wrapper and query generator. It provides a clean, functional interface for working with databases.

Why Ecto?

Ecto gives you:

• Type safety: Schemas define the structure of your data
• Query building: Functional query interface
• Changesets: Data validation and transformation
• Migrations: Database schema management
• Multiple databases: Support for PostgreSQL, MySQL, SQLite, and more

Setting up Ecto

First, add Ecto to your dependencies:

# mix.exs
defp deps do
  [
    {:ecto_sql, "~> 3.10"},
    {:postgrex, ">= 0.0.0"}
  ]
end

Then configure your database:

# config/dev.exs
config :my_app, MyApp.Repo,
  username: "postgres",
  password: "postgres",
  hostname: "localhost",
  database: "my_app_dev",
  stacktrace: true,
  show_sensitive_data_on_connection_error: true,
  pool_size: 10

Schema Definition

Schemas define the structure of your data:

defmodule MyApp.Users.User do
  use Ecto.Schema
  import Ecto.Changeset

  schema "users" do
    field :name, :string
    field :email, :string
    field :age, :integer
    field :is_active, :boolean, default: true

    timestamps()
  end

  def changeset(user, attrs) do
    user
    |> cast(attrs, [:name, :email, :age, :is_active])
    |> validate_required([:name, :email])
    |> validate_format(:email, ~r/@/)
    |> validate_number(:age, greater_than: 0)
    |> unique_constraint(:email)
  end
end

The schema macro defines the database table structure, while the changeset function handles validation and data transformation.

Repository Operations

The repository is your interface to the database:

defmodule MyApp.Users do
  import Ecto.Query
  alias MyApp.Repo
  alias MyApp.Users.User

  def list_users do
    Repo.all(User)
  end

  def get_user!(id) do
    Repo.get!(User, id)
  end

  def create_user(attrs \\ %{}) do
    %User{}
    |> User.changeset(attrs)
    |> Repo.insert()
  end

  def update_user(%User{} = user, attrs) do
    user
    |> User.changeset(attrs)
    |> Repo.update()
  end

  # Complex queries
  def active_users do
    User
    |> where([u], u.is_active == true)
    |> order_by([u], u.name)
    |> Repo.all()
  end
end

The repository provides functions like all/1, get!/2, insert/1, update/1, and delete/1 for basic operations. For complex queries, you use the query interface.

Deploying Elixir Applications

Elixir makes deployment straightforward with releases. A release packages your application with the Erlang runtime, making it self-contained.

Creating Releases

# Create a release
mix release

# Create a release with hot code updates
mix release --overwrite

# Run the release
_build/prod/rel/my_app/bin/my_app start

Releases are great because:

• They're self-contained (no need to install Elixir on the server)
• They support hot code updates (update without downtime)
• They're optimized for production

Environment Configuration

Configure your application for different environments:

# config/runtime.exs
import Config

if config_env() == :prod do
  config :my_app, MyApp.Repo,
    url: System.get_env("DATABASE_URL"),
    pool_size: String.to_integer(System.get_env("POOL_SIZE") || "10")

  config :my_app, MyAppWeb.Endpoint,
    url: [host: System.get_env("HOST"), port: String.to_integer(System.get_env("PORT") || "4000")],
    http: [
      port: String.to_integer(System.get_env("PORT") || "4000")
    ],
    secret_key_base: System.get_env("SECRET_KEY_BASE")
end

The runtime.exs file is evaluated at runtime, allowing you to use environment variables for configuration.

What's Next?

You're getting close to mastering Elixir! In the final article, we'll explore:

• Advanced OTP patterns and behaviors
• Distributed Elixir and clustering
• Building microservices with Elixir
• Best practices and design patterns
• Real-world project examples

Practice Exercises

Try these to reinforce what you've learned:

1. GenServer: Create a simple key-value store using GenServer
2. Supervisor: Build a supervisor that manages multiple worker processes
3. Phoenix: Create a simple blog application with CRUD operations
4. Ecto: Implement a user management system with database persistence

Key Points

• OTP provides robust abstractions for building concurrent, fault-tolerant applications
• GenServer is the foundation for stateful processes in Elixir
• Supervisors ensure your applications can recover from failures
• Phoenix enables high-performance, real-time web applications
• Ecto provides a powerful and flexible database abstraction
• Elixir releases make deployment straightforward and reliable
• The combination of OTP and functional programming makes Elixir ideal for distributed systems