Of course Docker is a fantastic tool that allows us to work more efficiently and that offers new perspectives in terms of scalability, infrastructures, deployments,…
But first of all, Docker is used by a lot of people for development with many different languages on many different platforms.
And for developers (like me), the use of Docker is not something very easy nor fun if we have to type or copy/paste or run commands like…
docker run -it --rm \
-v $(pwd):/app \
-e VIRTUAL_HOST=domain.tld \
IMAGE_NAME \
/bin/bash -ci 'app/console cache:clear'
…to execute a simple Symfony command for example.
I’m sure Symfony developers understand what I mean regarding how many times a day we can run this command.
So it’s absolutely necessary to use Docker through the power of complementary tools such as:
- Shell scripting
- Make / Makefile
- Docker Compose
Shell Scripting
With your shell you will be able to simplify everything you need to use docker with no pain writing shell scripts. But writing such scripts is not always really easy for developers without admin or DevOps skills.
To show you how to be more productive with Docker and shell scripts, I’m gonna speak about the way I developed this Jekyll blog and how I work with it everyday.
A simple DO-FILE
You can see this full do.sh
file here.
And there it is the explanation of each part of the file:
Variables
Comments are explicit enough I think…
# Output colors
NORMAL="\\033[0;39m"
RED="\\033[1;31m"
BLUE="\\033[1;34m"
# Names to identify images and containers of this app
IMAGE_NAME='docker-ypereirareis'
CONTAINER_NAME="jekyll-ypereirareis"
# Usefull to run commands as non-root user inside containers
USER="bob"
HOMEDIR="/home/$USER"
EXECUTE_AS="sudo -u bob HOME=$HOME_DIR"
The last line of this part, EXECUTE_AS="sudo -u bob HOME=$HOME_DIR"
allows to execute commands into docker container as the non-root bob user
created in the image as you can see in my Dockerfile.
Logging functions
- The first one to log debug or info…
- The other one to log errors
log() {
echo "$BLUE > $1 $NORMAL"
}
error() {
echo ""
echo "$RED >>> ERROR - $1$NORMAL"
}
The Docker image
To run docker containers, you always need a docker image. An image is built thanks to a Dockerfile. It’s really important to be able to rebuild the image easily (upgrades,…).
build() {
docker build -t $IMAGE_NAME .
[ $? != 0 ] && \
error "Docker image build failed !" && exit 100
}
The shortcut functions
Each function is designed to execute an action in the installation or development process. Reading these functions you will see that my Jekyll project is built with:
- Node/npm
- Bower
- Jekyll (of course)
- Grunt
npm() {
log "NPM install"
docker run -it --rm -v $(pwd):/app $IMAGE_NAME \
bash -ci "$EXECUTE_AS npm install"
[ $? != 0 ] && error "Npm install failed !" && exit 101
}
bower() {
log "Bower install"
docker run -it --rm -v $(pwd):/app -v /var/tmp/bower:$HOMEDIR/.bower $IMAGE_NAME \
/bin/bash -ci "$EXECUTE_AS bower install \
--config.interactive=false \
--config.storage.cache=$HOMEDIR/.bower/cache"
[ $? != 0 ] && error "Bower install failed !" && exit 102
}
jkbuild() {
log "Jekyll build"
docker run -it --rm -v $(pwd):/app $IMAGE_NAME \
/bin/bash -ci "$EXECUTE_AS jekyll build"
[ $? != 0 ] && error "Jekyll build failed !" && exit 103
}
grunt() {
log "Grunt build"
docker run -it --rm -v $(pwd):/app $IMAGE_NAME \
/bin/bash -ci "$EXECUTE_AS grunt"
[ $? != 0 ] && error "Grunt build failed !" && exit 104
}
jkserve() {
log "Jekyll serve"
docker run -it -d --name="$CONTAINER_NAME" -p 4000:4000 -v $(pwd):/app $IMAGE_NAME \
/bin/bash -ci "jekyll serve -H 0.0.0.0"
[ $? != 0 ] && error "Jekyll serve failed !" && exit 105
}
The full installation function
When you start working on a new or an existing project, it’s really handy to have a simple command to execute to install and run a fully working application.
The following function will call every previous explained ones in the correct order.
install() {
echo "Installing full application at once"
remove
npm
bower
jkbuild
grunt
jkserve
}
Image/containers management
When working with docker, it’s important to be able to check what happens with containers or images:
- Log into a container
- Stop a specific container by name
- Start a specific container
- Remove a specific container
bash() {
log "BASH"
docker run -it --rm -v $(pwd):/app $IMAGE_NAME /bin/bash
}
stop() {
docker stop $CONTAINER_NAME
}
start() {
docker start $CONTAINER_NAME
}
remove() {
log "Removing previous container $CONTAINER_NAME" && \
docker rm -f $CONTAINER_NAME &> /dev/null || true
}
Help function
Expose the available functions through a simple help
function:
help() {
echo "-----------------------------------------------------------------------"
echo " Available commands -"
echo "-----------------------------------------------------------------------"
echo -e -n "$BLUE"
echo " > build - To build the Docker image"
echo " > npm - To install NPM modules/deps"
echo " > bower - To install Bower/Js deps"
echo " > jkbuild - To build Jekyll project"
echo " > grunt - To run grunt task"
echo " > jkserve - To serve the project/blog on 127.0.0.1:4000"
echo " > install - To execute full install at once"
echo " > stop - To stop main jekyll container"
echo " > start - To start main jekyll container"
echo " > bash - Log you into container"
echo " > remove - Remove main jekyll container"
echo " > help - Display this help"
echo -e -n "$NORMAL"
echo "-----------------------------------------------------------------------"
}
Run these functions
A shell script is generally executed with the following syntax:
./do.sh
But simply doing this, you will execute script instructions but you’ll never call your functions.
The last line $*
of the do.sh
script is VERY important.
It allows to call a function based on argument(s) passed to the script.
./do.sh install
for instance will execute the install function.
With a do-file you will work with docker writing simple shell commands like:
./do.sh build
./do.sh install
./do.sh grunt
- …
What about an alias for ./do.sh
in your .bashrc/.zshrc ?
Make / Makefile
If you’re not familiar with shell scripting you can choose another tool… a Makefile
.
Makefiles
are files used to configure make
, which is a build tool.
The principle is simple: to build a target, we indicate the dependencies and the command to build it.
make
is in charge of traveling the tree to build targets in the correct order.
helloyou: hello you end
cat hello.txt you.txt end.txt > helloyou.txt
cat helloyou.txt
hello:
echo "Hello" > hello.txt
you:
echo "you" > you.txt
end:
echo "!!!" > end.txt
A big advantage of make
and Makefiles
is that it provides auto complete out of the box.
Many people like using Makefile
in an uncommon way to run grouped commands like this for instance:
install: composer database cc
database:
app/console doctrine:database:create
app/console doctrine:schema:create
app/console doctrine:schema:update
composer:
composer install --prefer-source
cc:
app/console cache:clear --env=prod
We could replace every symfony command with a docker run
command just like in our do-file:
install: composer database cc
database:
docker run --rm -it -v $(pwd):/app $IMAGE_NAME \
/bin/bash -ci "app/console doctrine:database:create"
docker run --rm -it -v $(pwd):/app $IMAGE_NAME \
/bin/bash -ci "app/console doctrine:schema:create"
docker run --rm -it -v $(pwd):/app $IMAGE_NAME \
/bin/bash -ci "app/console doctrine:schema:update"
composer:
docker run --rm -it -v $(pwd):/app $IMAGE_NAME \
/bin/bash -ci "composer install --prefer-dist"
cc:
docker run --rm -it -v $(pwd):/app $IMAGE_NAME \
/bin/bash -ci "app/console cache:clear --env=prod"
Be careful… When using Makefile
you must use tab key/character in your commands.
Docker Compose
Docker compose is not a third way to work more easily with docker
commands,
it is a tool allowing simple containers orchestration.
Compose is a tool for defining and running complex applications with Docker. With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.
Compose allows you to define the services/containers that make up your app in a docker-compose.yml
file, so they can be run together in an isolated environment.
Example:
web:
build: docker
working_dir: /var/www
hostname: project
domainname: project.dev
command: /root/start.sh
volumes:
- ".:/var/www"
- "./var/logs/web:/var/log/apache2"
environment:
VIRTUAL_HOST: project.dev
SYMFONY__APP__SECRET: secret
SYMFONY__CRYPT__KEY: key
SYMFONY__DATABASE__HOST: db
SYMFONY__DATABASE__PORT: 3306
SYMFONY__DATABASE__NAME: database
SYMFONY__DATABASE__USER: root
SYMFONY__DATABASE__PASSWORD: database password
links:
- db
db:
image: mysql:5.5
environment:
MYSQL_ROOT_PASSWORD: SUP3R-STR0NG-P@SSW0RD
volumes:
- "./data:/var/lib/mysql"
composer:
image: zolweb/docker-composer
working_dir: /src
volumes:
- /var/tmp/composer:/root/.composer
- ".:/src"
net: "host"
Then run docker-compose up
and docker compose will start your entire app with correct links, environment variables,…
Since docker compose 1.2 you can now define configurations for multiple environments thanks to the extends
keyword.
docker-common.yml
web:
build: docker
working_dir: /var/www
hostname: project
domainname: project.dev
command: /root/start.sh
volumes:
- ".:/var/www"
- "./var/logs/web:/var/log/apache2"
environment:
VIRTUAL_HOST: project.dev
SYMFONY__APP__SECRET: secret
SYMFONY__DATABASE__HOST: db
SYMFONY__DATABASE__PORT: 3306
SYMFONY__DATABASE__NAME: database
SYMFONY__DATABASE__USER: root
SYMFONY__DATABASE__PASSWORD: database password
db:
image: mysql:5.5
environment:
MYSQL_ROOT_PASSWORD: SUP3R-STR0NG-P@SSW0RD
volumes:
- "./data:/var/lib/mysql"
composer:
image: zolweb/docker-composer
working_dir: /src
volumes:
- /var/tmp/composer:/root/.composer
- ".:/src"
net: "host"
Links can’t be inherited and must appear in “per environment” files docker-compose.yml
and docker-prod.yml
.
docker-compose.yml
web:
extends:
file: docker-compose-common.yml
service: web
links:
- db
environment:
SYMFONY__CRYPT__KEY: DEV_CRYPT_KEY
db:
extends:
file: docker-compose-common.yml
service: db
composer:
extends:
file: docker-compose-common.yml
service: composer
You must extend every service/container used in your “environment specific” config.
docker-prod.yml
web:
extends:
file: docker-compose-common.yml
service: web
links:
- db
environment:
SYMFONY__CRYPT__KEY: PROD_CRYPT_KEY
SYMFONY__PROD__HOSTNAME: PROD_HOST_NAME
db:
extends:
file: docker-compose-common.yml
service: db
environment:
MYSQL_ROOT_PASSWORD: SUP3R-PR0DUCT10N-P@SSW0RD
composer:
extends:
file: docker-compose-common.yml
service: composer
To run docker compose with another file than the default one docker-compose.yml
use the -f
option:
docker-compose -f docker-prod.yml up
Conclusion
Choose a solution and a set of tools that fits your needs.
But, please, do not run docker
commands directly from your shell without any complementary tool anymore.