git diff

git diff --name-only  tag1 tag2
------
src/test1.dicon
src/test2.dicon
src/test3.dicon

git diff -w --name-status --no-renames tag1 tag2 >file-name-status.txt
MODE |  PATH
----  |---------
M src/test1.dicon
A src/test2.dicon
D src/test3.dicon

git diff -w --numstat --no-renames --diff-filter=M  tag1 tag2 

 > file-numstat.txt
ADD | DEL | PATH
----  |----  |-------
1 1 src/test1.dicon
1 0 src/test2.dicon

SSL/TLS通信仕組み概要

1. TLSはSSLの最新版、SSL2.0には脆弱性があるため、TLSに更新された、今のブラウザSSLを廃棄しTLSだけをサポートするのが多いです。

2. SSLの名前が広がるので、SSLの実態はTLSを指しています。

3. SSL/TLS通信の仕組み:

 公開キー・私有キーを利用し共通キーを受け渡し,共通キーでデータを暗号・複号し通信する.

    Client   ---------   request   ------------> server (public.key,  private.key)

    Client   <--------- public.key -----------  server  (public.key,  private.key)

    Client   

　　　public.keyを検証、信頼できる組織であるか、CAで保証する。
　　　信頼できる場合、
　　　　共通キー(A->c,B->d,C->e...)を作成し、public.keyで暗号化し
         ---------------------------------------> server  (public.key,  private.key)

                                       private.keyで共通キー(A->c,B->d,C->e...)を復号

　 Client(共通キー) <-------共通キーで暗号化済response----- server  (共通キー)

    Client(共通キー)  -------共通キーで暗号化済request-------- server  (共通キー)

　 Client(共通キー) <-------共通キーで暗号化済response----- server  (共通キー)

    ......

netstat show network info

netstat  -abno | grep 80
プロトコルの統計と現在の TCP/IP ネットワーク接続を表示し

• -a shows all connections and listening ports.
• -b shows the executable responsible for each connection or listening port.
• -n shows the IP address and port numbers numerically.
• -o shows the process ID for each connection or listening port.

netstat --help

プロトコルの統計と現在の TCP/IP ネットワーク接続を表示します。

NETSTAT [-a] [-b] [-e] [-f] [-n] [-o] [-p proto] [-r] [-s] [-t] [-x] [-y] [interval]

  -a            すべての接続とリッスン ポートを表示します。

  -b            それぞれの接続またはリッスン ポートの作成に使われた実行可能

                ファイルを表示します。場合により、よく知られた実行可能ファイルが

                複数の独立したコンポーネントをホストすることもあり、その

                場合、接続またはリッスン ポートの作成に使われたコンポーネント

                群が表示されます。この場合、実行可能ファイル名は下に [ ] で表示

                され、上には TCP/IP に到達するまで順に呼び出したコンポーネント

                が表示されます。このオプションには時間がかかり、十分なアクセス

                許可がないとエラーが発生することに

                注意してください。

  -e           イーサネットの統計を表示します。これは、-s オプションと

                併用できます。

  -f            外部アドレスの完全修飾ドメイン名 (FQDN) を表示します。

  -n            アドレスとポート番号を数値形式で表示します。

  -o            各接続に関連付けられた各接続を所有するプロセス ID を表示します。

  -p proto      proto で指定されたプロトコルの接続を表示します。

                proto は次のいずれかです: TCP、UDP、TCPv6、UDPv6。プロトコルごとの

                統計を表示するために -s オプションと併用する場合、proto は次のいずれかです:IP、IPv6、ICMP、ICMPv6、TCP、TCPv6、UDP、UDPv6。

  -q            すべての接続、リッスン ポート、およびバインドされている

                非リッスン TCP ポートを表示します。バインドされている非リッスン ポートは、

                アクティブな接続に関連付けられている場合と関連付けられていない場合があります。

  -r            ルーティング テーブルを表示します。

  -s            プロトコルごとの統計を表示します。既定では、IP、IPv6、ICMP、

                ICMPv6、TCP、TCPv6、UDP、UDPv6 の統計が表示されます。

                -p オプションを使って、既定のプロトコルのサブセットを指定できます。

  -t            現在の接続のオフロード状態を表示します。

  -x            NetworkDirect の接続、リスナー、共有エンドポイントを表示します。

  -y            すべての接続の TCP 接続テンプレートを表示します。

                他のオプションとの併用はできません。

  interval      各表示で interval 秒間一時停止しながら、選択した統計を再表示

                します。統計の再表示を停止するには、Ctrl+C を押します。

                interval が省略された場合、現在の構成情報を

                一度だけ表示します。

windows/linux SHA256

REM: windows: CertUtil -hashfile testfile.zip SHA256

#linux sha256sum testfile.zip

minikube basic

Reference: https://minikube.sigs.k8s.io/docs/handbook/controls/

 Start a cluster by running:

minikube start

Access the Kubernetes dashboard running within the minikube cluster:

minikube dashboard

Once started, you can interact with your cluster using kubectl, just like any other Kubernetes cluster. For instance, starting a server:

# create deployment
kubectl create deployment regex-minikube --image=registry.dev.mbpsmartec.co.jp/regexgo:v4
# expose a service as a NodePort
kubectl expose deployment regex-minikube --type=NodePort --port=8080

minikube makes it easy to open this exposed endpoint in your browser:

minikube service regex-minikube

Upgrade your cluster:

minikube start --kubernetes-version=latest

Start a second local cluster
(note: This will not work if minikube is using the bare-metal/none driver):

minikube start -p cluster2

Deploying apps
kubectl create deployment hello-minikube1 --image=kicbase/echo-server:1.0
kubectl expose deployment hello-minikube1 --type=LoadBalancer --port=8080
kubectl expose deployment hello-minikube1 --type=NodePort --port=8080

$ kubectl get svc
NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
hello-minikube1   NodePort    10.100.238.34   <none>        8080:31389/TCP   3s

$ minikube service hello-minikube1 --url

Addons (Istio or Ingress) built-in list of applications and services
minikube addons list 
minikube addons enable <name>
#enable an addon at start-up, where –addons option can be specified multiple times:
minikube start --addons <name1> --addons <name2>

Stop your local cluster:

minikube stop

Delete your local cluster:

minikube delete

#Delete all local clusters and profiles
minikube delete --all

kind(kube in docker) instroduction

Linux:

______________________________________

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

#check install 

kubectl version --client

curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.10.0/kind-linux-amd64

chmod +x ./kind

sudo mv kind /usr/local/bin/

#check install 

kind version

__________________________________________

kind create cluster --name=kind2 --config=kind-config.yaml

-------------------------kind-config.yaml: 

# this config file contains all config fields with comments

# NOTE: this is not a particularly useful config file

kind: Cluster

apiVersion: kind.x-k8s.io/v1alpha4

# patch the generated kubeadm config with some extra settings

kubeadmConfigPatches:

- |

  apiVersion: kubelet.config.k8s.io/v1beta1

  kind: KubeletConfiguration

  evictionHard:

    nodefs.available: "0%"

# patch it further using a JSON 6902 patch

kubeadmConfigPatchesJSON6902:

- group: kubeadm.k8s.io

  version: v1beta2

  kind: ClusterConfiguration

  patch: |

    - op: add

      path: /apiServer/certSANs/-

      value: my-hostname

# 1 control plane node and 3 workers

nodes:

# the control plane node config. default one node hosting a control plane

- role: control-plane

  kubeadmConfigPatches:

  - |

    kind: InitConfiguration

    nodeRegistration:

      kubeletExtraArgs:

        node-labels: "my-label=true"

  extraMounts:

  - hostPath: /home/test-user/data/files/

    containerPath: /files

  extraPortMappings:

  - containerPort: 80

    hostPort: 8000

    listenAddress: "0.0.0.0" # Optional, defaults to "0.0.0.0"

    #protocol: udp # Optional, defaults to tcp

# the three workers

- role: worker

- role: worker

- role: worker

linux cat create a txt file by EOF

EOF is a token that tells the cat command to terminate when it sees such a token in the subsequent lines.

cat > readme.txt << EOF

This is an input stream literal EOF

Followed by CTRL+D to terminate the command:

cat > readme.txt This is a readme file. This is a new line.

 

Append to an existing file, we can use the “>>” operator:

cat >> readme.txt
This is an appended line.

powershell set-alias grep select-string

Destination: using grep in powershell like docker ps | grep "container-name"  

1. Powershell set-alias:
    Set-Alias -name grep -value Select-String

2. Try: docker ps -a | grep kind

PS C:\test> help select-string

名前

    Select-String

概要

    Finds text in strings and files.

構文

    Select-String [-Pattern] <System.String[]> [-AllMatches] [-CaseSensitive] [-Context <System.Int32[]>] [-Encoding {ASCII | BigEndianUnicode | Default | OEM | Unicode | UTF7 | UTF8 | UTF32}] [-Exclude <System.String[]>] [-Include <System.String[]>] -InputObject <System.Management.Autom

    ation.PSObject> [-List] [-NotMatch] [-Quiet] [-SimpleMatch] [<CommonParameters>]

    Select-String [-Pattern] <System.String[]> [-AllMatches] [-CaseSensitive] [-Context <System.Int32[]>] [-Encoding {ASCII | BigEndianUnicode | Default | OEM | Unicode | UTF7 | UTF8 | UTF32}] [-Exclude <System.String[]>] [-Include <System.String[]>] [-List] -LiteralPath <System.String[]

    > [-NotMatch] [-Quiet] [-SimpleMatch] [<CommonParameters>]

    Select-String [-Pattern] <System.String[]> [-Path] <System.String[]> [-AllMatches] [-CaseSensitive] [-Context <System.Int32[]>] [-Encoding {ASCII | BigEndianUnicode | Default | OEM | Unicode | UTF7 | UTF8 | UTF32}] [-Exclude <System.String[]>] [-Include <System.String[]>] [-List] [-NotMatch] [-Quiet] [-SimpleMatch] [<CommonParameters>]

説明

    The `Select-String` cmdlet uses regular expression matching to search for text patterns in input strings and files. You can use `Select-String

    ` similar to `grep` in UNIX or `findstr.exe` in Windows.

    `Select-String` is based on lines of text. By default, `Select-String` finds the first match in each line and, for each match, it displays the

     file name, line number, and all text in the line containing the match. You can direct `Select-String` to find multiple matches per line, disp

    lay text before and after the match, or display a Boolean value (True or False) that indicates whether a match is found.

    `Select-String` can display all the text matches or stop after the first match in each input file. `Select-String` can be used to display all

    text that doesn't match the specified pattern.

    You can also specify that `Select-String` should expect a particular character encoding, such as when you're searching files of Unicode text.

    `Select-String` uses the byte-order-mark (BOM) to detect the encoding format of the file. If the file has no BOM, it assumes the encoding is U

    TF8.

関連するリンク

    Online Version: https://docs.microsoft.com/powershell/module/microsoft.powershell.utility/select-string?view=powershell-5.1&WT.mc_id=ps-gethel

    p

    about_Automatic_Variables

    about_Comparison_Operators

    about_Functions

    about_Quoting_Rules

    about_Regular_Expressions

    Get-Alias

    Get-ChildItem

    Get-Command

-- More  --

DB Performance Optimize

 PostgrelDB as example.

1. Install PostgrelSQL with default settings.
    with Selected Type Of cloud instance(Hardwares/OS/DB) as Standard. ex:
     

2. Check the performance insights counter metrics on AWS CloudWatch before.

3. Change the setting and check the performance  
    - memory: 

4. Change the setting and check the performance  
    - disk cache:  8MB->128MB

5. Change the Transaction log buffer and check the performance
    - wal_buffers: 16 -> 32

6. Change the Backgroud Write Process Count and check the performance
   - bgwrite_delay: 200
   - bgwrite_maxpages: 4 ->8

7. Change the checkpoint_segements and check the performance
   - checkpoint_segments: 4->16

8. Turn off recording disk access time and check the performance(Linux only)
   - mount -o noatime /etc/fstab

9. Change tablespace saved disk difference to pg_default.
    CREATE TABLESPACE pgdata1 LOCATION '/disk1/pgdata1';
    CREATE DATABASE ～ TABLESPACE pgdata1;
    CREATE TABLE ～ TABLESPACE pgdata1;
    ALTER TABLE ～ SET TABLESPACE pgdata1;

10. Clean the trash in disk monthly.

Introduce kubernetes, kubectl, and minikube as a local tool.

Kubernetes

Kubernetes is a production-grade, open-soure platform that orchestrates the placement(scheduling) and execution of application containers within and across computer clusters. 

Kubernetes clusters 

kubernetes coordinates a highly available cluster of computers that are connected to work as a single unit.  

Kubernetes automates the distribution and scheduling of application containers across a cluster in a more efficient way.

A kubernetes cluster consists of two types of resources:

• The Control Plane coordinates the cluster
• Nodes are the workers that run applications

kubectl

The Kubernetes command-line tool, allows you to run commands against Kubernetes clusters. 

You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs. For more information including a complete list of kubectl operations, see the kubectl reference documentation.

minikube

A tool that lets you run Kubernetes locally.

minikube runs an all-in-one or a multi-node local Kubernetes cluster on your personal computer (including Windows, macOS and Linux PCs) so that you can try out Kubernetes, or for daily development work.

You can follow the official Get Started! guide if your focus is on getting the tool installed.

Once you have minikube working, you can use it to run a sample application.

Introduction of minikube kubectl cluster pod service

0.1. Download and install Docker desktop. 

0.2. Download and install minikube.

     windows / Linux / docker  

Reference : https://kubernetes.io/docs/tutorials/kubernetes-basics/

Kubernetes Basics Modules

Pods that are running inside Kubernetes are running on a private, isolated network. By default they are visible from other pods and services within the same kubernetes cluster, but not outside that network. 

 A Pod is a Kubernetes abstraction that represents a group of one or more application containers (such as Docker), and some shared resources for those containers. Those resources include:

• Shared storage, as Volumes
• Networking, as a unique cluster IP address
• Information about how to run each container, such as the container image version or specific ports to use

A Pod models an application-specific "logical host" and can contain different application containers which are relatively tightly coupled.

Pods overview

A Pod always runs on a Node. A Node is a worker machine in Kubernetes and may be either a virtual or a physical machine, depending on the cluster. Each Node is managed by the control plane. A Node can have multiple pods, and the Kubernetes control plane automatically handles scheduling the pods across the Nodes in the cluster. The control plane's automatic scheduling takes into account the available resources on each Node.

A Node can have multiple pods, and the Kubernetes control plane automatically handles scheduling the pods across the Nodes in the cluster. 

Node overview

Services and Labels

A Service routes traffic across a set of Pods. Services are the abstraction that allows pods to die and replicate in Kubernetes without impacting your application. Discovery and routing among dependent Pods (such as the frontend and backend components in an application) are handled by Kubernetes Services.

Services match a set of Pods using labels and selectors, a grouping primitive that allows logical operation on objects in Kubernetes. Labels are key/value pairs attached to objects and can be used in any number of ways:

• Designate objects for development, test, and production
• Embed version tags
• Classify an object using tags

Services can be exposed in different ways by specifying a type in the spec of the Service:

• ClusterIP (default) - Exposes the Service on an internal IP in the cluster. This type makes the Service only reachable from within the cluster.
• NodePort - Exposes the Service on the same port of each selected Node in the cluster using NAT. Makes a Service accessible from outside the cluster using <NodeIP>:<NodePort>. Superset of ClusterIP.
• LoadBalancer - Creates an external load balancer in the current cloud (if supported) and assigns a fixed, external IP to the Service. Superset of NodePort.
• ExternalName - Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record with its value. No proxying of any kind is set up. This type requires v1.7 or higher of kube-dns, or CoreDNS version 0.0.8 or higher.

Create a new Service

> kubectl expose deployment/kubernetes-bootcamp --type="NodePort" --port 8080

> kubectl get services

> kubectl describe services/kubernetes-bootcamp

#query our list of Pods. We’ll use the kubectl get pods command with -l as a parameter
> kubectl get pods -l app=kubernetes-bootcamp

NAME                                   READY   STATUS    RESTARTS   AGE

kubernetes-bootcamp-5485cc6795-fj5v9   1/1     Running   0          5h7m

kubernetes-bootcamp-5485cc6795-qf8l8   1/1     Running   0          5h8m

#do the same to list the existing Services:
> kubectl get services -l app=kubernetes-bootcamp

NAME                  TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE

kubernetes-bootcamp   LoadBalancer   10.111.187.193   <pending>     8080:30790/TCP   4h2m

# apply a new label we use the label subcommand followed by the object type

kubectl label pods "$POD_NAME" version=v1
#query now the list of pods using the new label:

kubectl describe pods "$POD_NAME"
kubectl get pods -l version=v1

Deleting a service

kubectl delete service -l app=kubernetes-bootcamp

Previously we created a Deployment, and then exposed it publicly via a Service. The Deployment created only one Pod for running our application. When traffic increases, we will need to scale the application to keep up with user demand.

Scaling an application

Scaling out a Deployment will ensure new Pods are created and scheduled to Nodes with available resources. 
Scaling will increase the number of Pods to the new desired state.

To see the ReplicaSet created by the Deployment, run kubectl get rs




scale the Deployment to 4 replicas. We’ll use the kubectl scale command

kubectl scale deployments/kubernetes-bootcamp --replicas=4

To list your Deployments once again, use get deployments:

kubectl get deployments

The change was applied, and we have 4 instances of the application available. Next, let’s check if the number of Pods changed:

kubectl get pods -o wide

There are 4 Pods now, with different IP addresses. The change was registered in the Deployment events log. To check that, use the describe subcommand:

kubectl describe deployments/kubernetes-bootcamp

You can also view in the output of this command that there are 4 replicas now.

Scale Down

To scale down the Deployment to 2 replicas, run again the scale subcommand:

kubectl scale deployments/kubernetes-bootcamp --replicas=2

kubectl get deployments
kubectl get pods -o wide

Scaling overview

Performing a Rolling Update

for 

Updating an application

To update the image of the application to version 2, use the set image subcommand, followed by the deployment name and the new image version:

kubectl set image deployments/kubernetes-bootcamp kubernetes-bootcamp=jocatalin/kubernetes-bootcamp:v2
#deployment.apps/kubernetes-bootcamp image updated

The command notified the Deployment to use a different image for your app and initiated a rolling update. 

Verify an update

check that the app is running
kubectl describe services/kubernetes-bootcamp

#confirm the update by running the rollout status subcommand:

kubectl rollout status deployments/kubernetes-bootcamp




Let’s perform another update, and try to deploy an image tagged with v10:

kubectl set image deployments/kubernetes-bootcamp kubernetes-bootcamp=gcr.io/google-samples/kubernetes-bootcamp:v10

Use get deployments to see the status of the deployment:

kubectl get deployments

Notice that the output doesn't list the desired number of available Pods. Run the get pods subcommand to list all Pods:

kubectl get pods

Notice that some of the Pods have a status of ImagePullBackOff.

To get more insight into the problem, run the describe pods subcommand:
kubectl describe pods

In the Events section of the output for the affected Pods, notice that the v10 image version did not exist in the repository.

To roll back the deployment to your last working version, use the rollout undo subcommand:

kubectl rollout undo deployments/kubernetes-bootcamp 
#deployment.apps/kubernetes-bootcamp rolled back

Use the get pods subcommand to list the Pods again:

kubectl get pods

Four Pods are running. To check the image deployed on these Pods, use the describe pods subcommand:

kubectl describe pods




#clean up local cluster
kubectl delete deployments/kubernetes-bootcamp services/kubernetes-bootcamp

Troubleshooting with kubectl

• kubectl get - list resources
  kubectl get pods # like docker ps  
• kubectl describe - show detailed information about a resource
  kubectl describe pods # like docker inspect -it $container_name 
• kubectl logs - print the logs from a container in a pod
  kubectl logs "$POD_NAME"    # like docker logs -it $container_name
• kubectl exec - execute a command on a container in a pod
  kubectl exec -it $POD_NAME -- bash # like docker exec -it $container_name