create docker image from docker container by commit

1. Run Docker base image like ubuntu 

[node1] (local) root@192.168.0.23 ~
$ docker run -it ubuntu bash
root@153c1eb1d2c4:/# apt-get update 
Get:1 http://security.ubuntu.com/ubuntu 
focal-security InRelease [107 kB]
...
Reading package lists... Done
root@153c1eb1d2c4:/# apt-get install -y figlet
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  figlet
root@153c1eb1d2c4:/# figlet "hello docker"
 _          _ _           _            _
| |__   ___| | | ___   __| | ___   ___| | _____ _ __
| '_ \ / _ \ | |/ _ \ / _` |/ _ \ / __| |/ / _ \ '__|
| | | |  __/ | | (_) | (_| | (_) | (__|   <  __/ |
|_| |_|\___|_|_|\___/ \__,_|\___/ \___|_|\_\___|_|
root@153c1eb1d2c4:/# exit 
[node1] (local) root@192.168.0.23 ~
$ docker ps -a 
ONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                            PORTSNAMES
153c1eb1d2c4        ubuntu              "bash"              6 minutes ago       Exited (127) About a minute ago
$ docker commit --author lqwangxg --message "save as image" 153
sha256:7ff21c05ee7d57d44f7733e0d116080e0ac66c432836211403d82fd18c9ad36f
$ docker images 
EPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
<none>              <none>              7ff21c05ee7d        12 seconds ago      97.2MB
$ docker tag 7ff ubuntu-figlet
$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
ubuntu-figlet       latest              7ff21c05ee7d        2 minutes ago       97.2MB

Study memo of docker swarm

1. Docker swarm init 

$ docker swarm init --advertise-addr $(host -i)
docker swarm init --advertise-addr $(hostname -i)
Swarm initialized: current node (s19krjqmzoz0zozspjszjojdk) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-589slthktmh85s376jqypz4qtt8qyj7nbpjj6i840sgm0nfo2m-87sdp1mneat2kxiwo6tblag3c 192.168.0.13:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
   

2. Docker node ls

$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
s19krjqmzoz0zozspjszjojdk *   node1               Ready               Active              Leader              19.03.11   

3. Docker stack deploy

$ docker stack deploy --help 
Usage:  docker stack deploy [OPTIONS] STACK

Deploy a new stack or update an existing stack

Aliases:
  deploy, up

Options:
      --bundle-file string     Path to a Distributed Application Bundle file
  -c, --compose-file strings   Path to a Compose file, or "-" to read from stdin
      --orchestrator string    Orchestrator to use (swarm|kubernetes|all)
      --prune                  Prune services that are no longer referenced
      --resolve-image string   Query the registry to resolve image digest and supported platforms
                               ("always"|"changed"|"never") (default "always")
      --with-registry-auth     Send registry authentication details to Swarm agents

3. Docker stack deploy

$ docker stack deploy --help 
Usage:  docker stack deploy [OPTIONS] STACK

Deploy a new stack or update an existing stack

Aliases:
  deploy, up

Options:
      --bundle-file string     Path to a Distributed Application Bundle file
  -c, --compose-file strings   Path to a Compose file, or "-" to read from stdin
      --orchestrator string    Orchestrator to use (swarm|kubernetes|all)
      --prune                  Prune services that are no longer referenced
      --resolve-image string   Query the registry to resolve image digest and supported platforms
                               ("always"|"changed"|"never") (default "always")
      --with-registry-auth     Send registry authentication details to Swarm agents

$ docker stack deploy --compose-file=docker-stack.yml voting_stack 
Creating network voting_stack_backend
Creating network voting_stack_frontend
Creating network voting_stack_default
Creating service voting_stack_result
Creating service voting_stack_worker
Creating service voting_stack_visualizer
Creating service voting_stack_redis
Creating service voting_stack_db
Creating service voting_stack_vote

$ cat docker-stack.yml
version: "3"
# create required networks and volummes. 
# services, networks, volumes, all are duplications.
networks:
  frontend:
  backend:

volumes:
  db-data:

services:
  redis:
    image: redis:alpine
    networks:
      - frontend
    deploy:
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure
  db:
    image: postgres:9.4
    environment:
      POSTGRES_USER: "postgres"
      POSTGRES_PASSWORD: "postgres"
    volumes:
      - db-data:/var/lib/postgresql/data
    networks:
      - backend
    deploy:
      placement:
        constraints: [node.role == manager]
  vote:
    image: dockersamples/examplevotingapp_vote:before
    ports:
      - 5000:80
    networks:
      - frontend
    depends_on:
      - redis
    deploy:
      replicas: 2
      update_config:
        parallelism: 2
      restart_policy:
        condition: on-failure
  result:
    image: dockersamples/examplevotingapp_result:before
    ports:
      - 5001:80
    networks:
      - backend
    depends_on:
      - db
    deploy:
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure

  worker:
    image: dockersamples/examplevotingapp_worker
    networks:
      - frontend
      - backend
    depends_on:
      - db
      - redis
    deploy:
      mode: replicated
      replicas: 1
      labels: [APP=VOTING]
      restart_policy:
        condition: on-failure
        delay: 10s
        max_attempts: 3
        window: 120s
      placement:
        constraints: [node.role == manager]

  visualizer:
    image: dockersamples/visualizer:stable
    ports:
      - "8080:8080"
    stop_grace_period: 1m30s
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
    deploy:
      placement:
        constraints: [node.role == manager]

Docker-compose build dockerfile and run service

Ref 

https://github.com/lqwangxg/dockerstudy

STEP4:

1. Show folder and files by tree commad.

$ tree
.
├── README.md
├── api
│   ├── Dockerfile
│   ├── linkextractor.py
│   ├── main.py
│   └── requirements.txt
├── docker-compose.yml
└── www
    └── index.php

2. Show ./api/Dockerfile

$ cat api/Dockerfile
FROM       lqwangxg/python
LABEL      maintainer="lqwangxg@gmail.com"

WORKDIR    /app
COPY       requirements.txt /app/
RUN        pip install -r requirements.txt

COPY       *.py /app/
RUN        chmod a+x *.py

CMD        ["./main.py"]    

3. Show  docker-compose.yml

$ cat api/Dockerfile
version: '3'

services:
  api:
    image: study4
    build: ./api
    ports:
      - "5000:5000"
  web:
    image: php:7-apache
    ports:
      - "80:80"
    environment:
      - API_ENDPOINT=http://api:5000/api/
    volumes:
      - ./www:/var/www/html    

4. Build and run by docker-compose.yml

$ docker-compose up -d --build
Building api
Step 1/8 : FROM       lqwangxg/python
 ---> 6cf621cb1327
Step 2/8 : LABEL      maintainer="lqwangxg@gmail.com"
 ---> Using cache
 ---> 3514f593e3e4
Step 3/8 : WORKDIR    /app
 ---> Using cache
 ---> 32c31859c252
Step 4/8 : COPY       requirements.txt /app/
 ---> Using cache
 ---> 8352c208cfb7
Step 5/8 : RUN        pip install -r requirements.txt
 ---> Using cache
 ---> c61c2800b117
Step 6/8 : COPY       *.py /app/
 ---> Using cache
 ---> 9aeb4cc13650
Step 7/8 : RUN        chmod a+x *.py
 ---> Using cache
 ---> e8ee2eceb5e1
Step 8/8 : CMD        ["./main.py"]
 ---> Using cache
 ---> 85bb2729865d
Successfully built 85bb2729865d
Successfully tagged study4:latest
dockerstudy_web_1 is up-to-date
dockerstudy_api_1 is up-to-date 

5. Show Runing Container service

$ docker ps 
ONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS        PORTS                    NAMES
af1f3f14e5bf        php:7-apache        "docker-php-entrypoi…"   17 minutes ago      Up 17 minutes       0.0.0.0:80->80/tcp       dockerstudy_web_1
e5f9e49916ef        study4              "./main.py"              17 minutes ago      Up 17 minutes       0.0.0.0:5000->5000/tcp   dockerstudy_api_1 

6. Show docker images 

$ docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
study4              latest              85bb2729865d        18 minutes ago      127MB
lqwangxg/python     latest              6cf621cb1327        6 days ago          113MB
php                 7-apache            7a935726473b        6 days ago          414MB

7. Add redis service to  docker-compose.yml

$ cat api/Dockerfile
version: '3'

services:
  api:
    image: study4
    build: ./api
    ports:
      - "5000:5000"
    enviroment:
      - REDIS_URL=redis://redis:6379
  web1:
    image: php:7-apache
    ports:
      - "80:80"
    environment:
      - API_ENDPOINT=http://api:5000/api/
    volumes:
      - ./www:/var/www/html 
  web2:
    image: php:7-apache
    build: ./www 
    ports:
      - "81:80"
    environment:
      - API_ENDPOINT=http://api:5000/api
  redis:
    image: redis

# web1:  don't build docker image, run php:7-apache directly.
# web2:  create a new docker image from php:7-apache, and run. 

8. Build and run by docker-compose.yml

$ docker-compose up -d --build
Building api
Step 1/9 : FROM       lqwangxg/python
 ---> 6cf621cb1327
Step 2/9 : LABEL      maintainer="lqwangxg@gmail.com"
 ---> Using cache
 ---> 3514f593e3e4
Step 3/9 : ENV        REDIS_URL="redis://localhost:6379"
 ---> Running in 0dd195b593ba
Removing intermediate container 0dd195b593ba
 ---> a8f8264daa08
Step 4/9 : WORKDIR    /app
 ---> Running in be0262a54304
Removing intermediate container be0262a54304
 ---> bc1e8588280a
Step 5/9 : COPY       requirements.txt /app/
 ---> 411f5b4456e8
Step 6/9 : RUN        pip install -r requirements.txt
 ---> Running in 8ab8378a058b
Collecting beautifulsoup4
...
Building web
Step 1/4 : FROM       php:7-apache
 ---> 7a935726473b
Step 2/4 : LABEL      maintainer="Sawood Alam <@ibnesayeed>"
 ---> Running in f591f0536f20
Removing intermediate container f591f0536f20
 ---> bcd8bd2a2bf3
Step 3/4 : ENV        API_ENDPOINT="http://localhost:5000/api/"
 ---> Running in 2ee1abfdbdaf
Removing intermediate container 2ee1abfdbdaf
 ---> 2a6357cb298d
Step 4/4 : COPY       . /var/www/html/
 ---> c11b8ee6d1d7
Successfully built c11b8ee6d1d7
Successfully tagged study5php:latest
Pulling redis (redis:)...
latest: Pulling from library/redis
d121f8d1c412: Already exists
2f9874741855: Pull complete
d92da09ebfd4: Pull complete
bdfa64b72752: Pull complete
e748e6f663b9: Pull complete
eb1c8b66e2a1: Pull complete
Digest: sha256:1cfb205a988a9dae5f025c57b92e9643ec0e7ccff6e66bc639d8a5f95bba928c
Status: Downloaded newer image for redis:latest
Recreating dockerstudy_web_1 ... done
Recreating dockerstudy_api_1 ... done
Creating dockerstudy_redis_1 ... done
Creating dockerstudy_web2_1  ... done 

9. Docker compose down

$ docker compose down 
Stopping dockerstudy_api_1   ... done
Stopping dockerstudy_web_1   ... done
Stopping dockerstudy_web2_1  ... done
Stopping dockerstudy_redis_1 ... done
Removing dockerstudy_api_1   ... done
Removing dockerstudy_web_1   ... done
Removing dockerstudy_web2_1  ... done
Removing dockerstudy_redis_1 ... done
Removing network dockerstudy_default

create a hello world by vite in linux docker

. make folder and create-vite-app.sh

mkdir vite
cd vite 
cat <<EOF >create-vite-app.sh
appname=$1
if [ -z $appname ]; then 
  echo "appname is not set, using default hello-vite."
  appname=hello-vite
else
  echo "appname=$appname"
fi  

npm init vite-app $appname
cd $appname
npm install 
npm run dev
EOFchmod +x create-vite-app.sh

2. Create vite-app by  node:alpine  image

docker run -it --rm --name vite \
-w /app \
-v ~/vite:/app \
-p 3000:3000 \
lqwangxg/node:alpine \
./create-vite-app.sh vite-app-01

3.  Check to create the result.

ls 
create-vite-app.sh  vite-app-01
ls vite-app-01/
index.html         package-lock.json  public
node_modules       package.json       src

K8s On Docker desktop Manage deployment and service

1. Kubernetes create deployment and pod 

> kubectl get nodes 
NAME             STATUS   ROLES    AGE   VERSION
docker-desktop   Ready    master   62m   v1.16.6-beta.0

> kubectl run testnginx --image lqwangxg/nginx
kubectl run --generator=deployment/apps.v1 is DEPRECATED and 
will be removed in a future version. 
Use kubectl run --generator=run-pod/v1 or kubectl create instead.
deployment.apps/testnginx created

> kubectl get deployment 
NAME        READY   UP-TO-DATE   AVAILABLE   AGE
testnginx   1/1     1            1           62s

> kubectl get pods 
NAME                         READY   STATUS    RESTARTS   AGE
testnginx-78cc8cbd6c-7x4zb   1/1     Running   0          98s

2. Kubernetes expose deployment as service 

> kubectl expose deployment testnginx --port 80 --type NodePort
service/testnginx exposed

> kubectl get service 
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP        74m
testnginx    NodePort    10.106.146.174   <none>        80:32171/TCP   28s
> curl http://localhost
StatusCode        : 200
StatusDescription : OK
Content           : <!DOCTYPE html>
                    <html>
                    <head>
                    <title>Welcome to nginx!</title>
...
> curl http://localhost:32171

StatusCode        : 200
StatusDescription : OK
Content           : <!DOCTYPE html>
                    <html>
                    <head>
                    <title>Welcome to nginx!</title>
                    <style>
                        body {
                            width: 35em;
                            margin: 0 auto;
                            font-family: Tahoma, Verdana, Arial, sans-serif;
                        }
                    </style>
                    <...
> kubectl delete deployment testnginx 
deployment.apps "testnginx" deleted

> kubectdl delete service 
service "testnginx" deleted

Example of Multi-Stage Docker file builder

1. Multi Stage Dockerfile build (node.js vue.js helloworld)

  ref:https://github.com/lqwangxg/vuechat     Dockerfile

FROM lqwangxg/node:alpine AS builder
WORKDIR /vuechat
RUN npm install -g @vue/cli @vue/cli-service-global 
COPY package*.json /vuechat/
RUN npm install
COPY . /vuechat/
RUN npm run build

FROM nginx:alpine
COPY --from=builder /vuechat/dist /usr/share/nginx/html

2. Multi Stage Dockerfile build 

FROM alpine:latest as builder
RUN apk --no-cache add build-base

FROM builder as build1
COPY source1.cpp source.cpp
RUN g++ -o /binary source.cpp

FROM builder as build2
COPY source2.cpp source.cpp
RUN g++ -o /binary source.cpp

###=======================
FROM production_image_template
#...
RUN ... to make the destination Image

Introduction of Docker nginx

 1. Docker run nginx:alpine with --mount

## mkdir from folder in dockerhost before --mount,
$ mkdir -p /var/www
$ mkdir -p /var/nginx/conf
## -p :make parent directories as needed.

## run nginx. Be careful:no space on every line head.
docker run --name mynginx2 \
--mount type=bind,\
source=/var/www,\
target=/usr/share/nginx/html,readonly \
--mount type=bind,\
source=/var/nginx/conf,\
target=/etc/nginx/conf,readonly \
-dp 8082:80 nginx:alpine

volume path must be absolute.
like:
  -v ~/vuechat:/vuechat 

## static html,js,css will be saved in /var/www/ by docker host
## nginx config will be save in /var/nginx/conf by docker host 
## Readonly from docker container. and read-write from dock host

 2. Docker Log Saved path:
    /var/lib/docker/containers/container-ID/container-ID-json.log 

$ docker ps -a 
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                  NAMES
c7f35ce5ec0b        nginx:alpine        "/docker-entrypoint.…"   15 minutes ago      Up 15 minutes       0.0.0.0:8082->80/tcp   mynginx2

$ ls /var/lib/docker/containers
c7f35ce5ec0b78abac3b19c6a31bac47282a774909f29406900609a30aaa6248

$ ls /var/lib/docker/containers/c7f*
c7f35ce5ec0b78abac3b19c6a31bac47282a774909f29406900609a30aaa6248-json.log
checkpoints
config.v2.json
hostconfig.json
hostname
hosts
mounts
resolv.conf
resolv.conf.hash

$cat /var/lib/docker/containers/c7f*/c7f*.log
...

{"log":"/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration\n","stream":"stdout","time":"2020-09-14T07:32:48.734972228Z"}
{"log":"/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/\n","stream":"stdout","time":"2020-09-14T07:32:48.735004429Z"}
{"log":"/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh\n","stream":"stdout","time":"2020-09-14T07:32:48.737170543Z"}
{"log":"10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf\n","stream":"stdout","time":"2020-09-14T07:32:48.743271983Z"}
{"log":"10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf\n","stream":"stdout","time":"2020-09-14T07:32:48.758506583Z"}
{"log":"/docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh\n","stream":"stdout","time":"2020-09-14T07:32:48.758856885Z"}
{"log":"/docker-entrypoint.sh: Configuration complete; ready for start up\n","stream":"stdout","time":"2020-09-14T07:32:48.763013513Z"}

3. Docker nginx Dockerfile 

FROM nginx
RUN rm /etc/nginx/nginx.conf /etc/nginx/conf.d/default.conf
COPY content /usr/share/nginx/html
COPY conf /etc/nginx
VOLUME /var/nginx/log

Docker rm all container / Docker rmi all images

 1. Docker rm  all containers  

docker ps -a | grep -i exited | tail -n +2 | awk '{print $1 }' | xargs docker rm 

# grep -i exited: filter exited docker containers by grep.
# tail -n +2: start from line2, skip header line.
# awk '{print $1}' :print the containerID
# send to docker rm for delete 

docker images | tail -n +2 | awk '{print $3 }' | xargs docker rmi 

# tail -n +2: start from line2, skip header line.
# awk '{print $3}' :print the imageID
# send to docker rmi for delete 

Introduction Of CI/CD workflow on Node Application

WORKFLOW

1. SOURCE DOWNLOAD FROM GITHUB ( git clone URL)
2. UPDATE SOURCE AND GIT COMMIT  (git commit -m "comment")
3. BUILD SOURCE BY JENKINS 
4. BUILD DOCKER IMAGES BY DOCKER BUILD
5. DEPLOY DOCKER IMAGE 

 1. Download/Upload source from git 

git clone https://github.com/lqwangxg/vuechat 
git commit -m "comment"
git push -u origin master 

 2. Docker pull Jenkins  and run Jenkins

docker pull jenkins/jenkins 

docker run -dp 8080:8080 -p 50000:50000 --name jenkins \
 -v jenkins_home:/var/jenkins_home jenkins/jenkins
/var/jenkins_home/secrets/initialAdminPassword

# play with docker: 
#cat /var/lib/docker/volumes/jenkins_home/_data/secrets/initialAdminPassword

 3. Config Jenkins 

#1 Jenkins system configuration
    install build tool plugins like github, nodejs, docker
#2 project configuration
  setting username password for logining to respository
#3 action before and after building.
  - git fetch from git repository
  - npm install 
  - npm run build 
  - docker build -tag imagename -f Dockerfile.deploy  .
  - docker run  -d -p 8080:8080 imagename