Docker Multi-Stage Build Cache Optimization - Slow Builds Despite Caching

I'm using Docker multi-stage builds for my application, but the build cache isn't working efficiently. Each stage rebuilds completely even when only small changes are made, resulting in slow builds. The cache layers aren't being reused properly between stages, and I'm seeing "CACHED" messages but the build still takes a long time. How can I optimize Docker multi-stage build caching?

Solution

The issue is likely caused by improper layer ordering and missing cache mount configurations in your multi-stage build. Here's how to optimize it:

Reorder layers from least to most frequently changing:

# Stage 1: Dependencies (changes least frequently)
FROM node:18-alpine AS dependencies
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production && npm cache clean --force

# Stage 2: Build tools (changes occasionally)
FROM node:18-alpine AS build-tools
WORKDIR /app
COPY package*.json ./
RUN npm ci

# Stage 3: Source code (changes most frequently)
FROM build-tools AS builder
COPY src/ ./src/
COPY public/ ./public/
COPY next.config.js ./
RUN npm run build

# Stage 4: Production (final stage)
FROM node:18-alpine AS production
WORKDIR /app
COPY --from=dependencies /app/node_modules ./node_modules
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/public ./public
COPY package*.json ./
EXPOSE 3000
CMD ["npm", "start"]

Use BuildKit cache mounts for better performance:

# syntax=docker/dockerfile:1
FROM node:18-alpine AS dependencies
WORKDIR /app
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci --only=production

FROM node:18-alpine AS build-tools
WORKDIR /app
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci

FROM build-tools AS builder
COPY src/ ./src/
COPY public/ ./public/
COPY next.config.js ./
RUN --mount=type=cache,target=/app/.next/cache \
    npm run build

Key optimizations:

Order layers by change frequency
Use --mount=type=cache for package managers
Separate dependency installation from source code copying
Use specific base images for each stage
Leverage BuildKit's advanced caching features

This approach significantly improves cache hit rates and build performance.

Alternative #1

If you're dealing with complex dependency trees or multiple package managers, you can use a more sophisticated caching strategy:

# syntax=docker/dockerfile:1
FROM node:18-alpine AS base
WORKDIR /app
RUN apk add --no-cache libc6-compat

# Dependencies stage with multiple cache layers
FROM base AS dependencies
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci --only=production --frozen-lockfile

# Dev dependencies stage
FROM base AS dev-dependencies
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci --frozen-lockfile

# Build stage with source code
FROM dev-dependencies AS builder
COPY . .
RUN --mount=type=cache,target=/app/.next/cache \
    npm run build

# Production stage
FROM base AS production
COPY --from=dependencies /app/node_modules ./node_modules
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/public ./public
COPY package*.json ./
EXPOSE 3000
CMD ["npm", "start"]

This approach:

Separates production and development dependencies
Uses frozen lockfile for reproducible builds
Maintains separate cache layers for different purposes
Reduces final image size by excluding dev dependencies

Alternative #2

For Java/Maven projects, you can optimize the build process by leveraging Maven's dependency resolution:

# syntax=docker/dockerfile:1
FROM maven:3.9-openjdk-17 AS dependencies
WORKDIR /app
COPY pom.xml .
RUN --mount=type=cache,target=/root/.m2 \
    mvn dependency:go-offline -B

FROM maven:3.9-openjdk-17 AS builder
WORKDIR /app
COPY pom.xml .
COPY src/ ./src/
RUN --mount=type=cache,target=/root/.m2 \
    mvn clean package -DskipTests

FROM openjdk:17-jre-slim AS production
WORKDIR /app
COPY --from=builder /app/target/*.jar app.jar
EXPOSE 8080
CMD ["java", "-jar", "app.jar"]

Key optimizations for Maven:

Use dependency:go-offline to download all dependencies first
Cache the .m2 directory across builds
Separate dependency resolution from compilation
Use slim JRE image for production
Skip tests during build (run them separately)

Alternative #3

If you're using Docker Compose or CI/CD pipelines, you can implement a distributed caching strategy:

# docker-compose.yml
version: '3.8'
services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
      cache_from:
        - myapp:latest
        - myapp:buildcache
      cache_to:
        - myapp:buildcache
    image: myapp:latest

# syntax=docker/dockerfile:1
FROM node:18-alpine AS base
WORKDIR /app

FROM base AS dependencies
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci --only=production

FROM base AS builder
COPY package*.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci
COPY . .
RUN --mount=type=cache,target=/app/.next/cache \
    npm run build

FROM base AS production
COPY --from=dependencies /app/node_modules ./node_modules
COPY --from=builder /app/.next ./.next
COPY --from=builder /app/public ./public
COPY package*.json ./
EXPOSE 3000
CMD ["npm", "start"]

This approach:

Uses registry-based caching for CI/CD
Shares cache between different builds
Works well with distributed build systems
Provides better cache persistence across environments