2023年7月31日星期一

DB Performance Optimize

 PostgrelDB as example.

1. Install PostgrelSQL with default settings.
    with Selected Type Of cloud instance(Hardwares/OS/DB) as Standard. ex:
     

2. Check the performance insights counter metrics on AWS CloudWatch before.

3. Change the setting and check the performance  
    - memory: 

4. Change the setting and check the performance  
    - disk cache:  8MB->128MB

5. Change the Transaction log buffer and check the performance
    - wal_buffers: 16 -> 32

6. Change the Backgroud Write Process Count and check the performance
   - bgwrite_delay: 200
   - bgwrite_maxpages: 4 ->8

7. Change the checkpoint_segements and check the performance
   - checkpoint_segments: 4->16

8. Turn off recording disk access time and check the performance(Linux only)
   - mount -o noatime /etc/fstab

9. Change tablespace saved disk difference to pg_default.
    CREATE TABLESPACE pgdata1 LOCATION '/disk1/pgdata1';
    CREATE DATABASE ~ TABLESPACE pgdata1;
    CREATE TABLE ~ TABLESPACE pgdata1;
    ALTER TABLE ~ SET TABLESPACE pgdata1;

10. Clean the trash in disk monthly.

2023年7月22日星期六

Introduce kubernetes, kubectl, and minikube as a local tool.

Kubernetes

Kubernetes is a production-grade, open-soure platform that orchestrates the placement(scheduling) and execution of application containers within and across computer clusters. 


Kubernetes clusters 

kubernetes coordinates a highly available cluster of computers that are connected to work as a single unit 

Kubernetes automates the distribution and scheduling of application containers across a cluster in a more efficient way.


A kubernetes cluster consists of two types of resources:
  • The Control Plane coordinates the cluster
  • Nodes are the workers that run applications

kubectl

The Kubernetes command-line tool, allows you to run commands against Kubernetes clusters. 
You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs. For more information including a complete list of kubectl operations, see the kubectl reference documentation.

minikube
A tool that lets you run Kubernetes locally.
minikube runs an all-in-one or a multi-node local Kubernetes cluster on your personal computer (including Windows, macOS and Linux PCs) so that you can try out Kubernetes, or for daily development work.
You can follow the official Get Started! guide if your focus is on getting the tool installed.
Once you have minikube working, you can use it to run a sample application.

2023年7月8日星期六

Introduction of minikube kubectl cluster pod service

0.1. Download and install Docker desktop. 

0.2. Download and install minikube.

     windows / Linux / docker  

Reference : https://kubernetes.io/docs/tutorials/kubernetes-basics/

Kubernetes Basics Modules

Pods that are running inside Kubernetes are running on a private, isolated network. By default they are visible from other pods and services within the same kubernetes cluster, but not outside that network

 A Pod is a Kubernetes abstraction that represents a group of one or more application containers (such as Docker), and some shared resources for those containers. Those resources include:

  • Shared storage, as Volumes
  • Networking, as a unique cluster IP address
  • Information about how to run each container, such as the container image version or specific ports to use
A Pod models an application-specific "logical host" and can contain different application containers which are relatively tightly coupled.

Pods overview

A Pod always runs on a Node. A Node is a worker machine in Kubernetes and may be either a virtual or a physical machine, depending on the cluster. Each Node is managed by the control plane. A Node can have multiple pods, and the Kubernetes control plane automatically handles scheduling the pods across the Nodes in the cluster. The control plane's automatic scheduling takes into account the available resources on each Node.

A Node can have multiple pods, and the Kubernetes control plane automatically handles scheduling the pods across the Nodes in the cluster. 

Node overview


Services and Labels

A Service routes traffic across a set of Pods. Services are the abstraction that allows pods to die and replicate in Kubernetes without impacting your application. Discovery and routing among dependent Pods (such as the frontend and backend components in an application) are handled by Kubernetes Services.

Services match a set of Pods using labels and selectors, a grouping primitive that allows logical operation on objects in Kubernetes. Labels are key/value pairs attached to objects and can be used in any number of ways:

  • Designate objects for development, test, and production
  • Embed version tags
  • Classify an object using tags

Services can be exposed in different ways by specifying a type in the spec of the Service:

  • ClusterIP (default) - Exposes the Service on an internal IP in the cluster. This type makes the Service only reachable from within the cluster.
  • NodePort - Exposes the Service on the same port of each selected Node in the cluster using NAT. Makes a Service accessible from outside the cluster using <NodeIP>:<NodePort>. Superset of ClusterIP.
  • LoadBalancer - Creates an external load balancer in the current cloud (if supported) and assigns a fixed, external IP to the Service. Superset of NodePort.
  • ExternalName - Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record with its value. No proxying of any kind is set up. This type requires v1.7 or higher of kube-dns, or CoreDNS version 0.0.8 or higher.

Create a new Service

> kubectl expose deployment/kubernetes-bootcamp --type="NodePort" --port 8080

> kubectl get services

> kubectl describe services/kubernetes-bootcamp

#query our list of Pods. We’ll use the kubectl get pods command with -l as a parameter
>
kubectl get pods -l app=kubernetes-bootcamp
NAME                                   READY   STATUS    RESTARTS   AGE
kubernetes-bootcamp-5485cc6795-fj5v9   1/1     Running   0          5h7m
kubernetes-bootcamp-5485cc6795-qf8l8   1/1     Running   0          5h8m

#do the same to list the existing Services:
> kubectl get services -l app=kubernetes-bootcamp
NAME                  TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
kubernetes-bootcamp   LoadBalancer   10.111.187.193   <pending>     8080:30790/TCP   4h2m

# apply a new label we use the label subcommand followed by the object type
kubectl label pods "$POD_NAME" version=v1
#query now the list of pods using the new label:
kubectl describe pods "$POD_NAME"
kubectl get pods -l version=v1

Deleting a service

kubectl delete service -l app=kubernetes-bootcamp


Previously we created a Deployment, and then exposed it publicly via a Service. The Deployment created only one Pod for running our application. When traffic increases, we will need to scale the application to keep up with user demand.

Scaling an application

Scaling out a Deployment will ensure new Pods are created and scheduled to Nodes with available resources. 
Scaling will increase the number of Pods to the new desired state.
To see the ReplicaSet created by the Deployment, run kubectl get rs

scale the Deployment to 4 replicas. We’ll use the kubectl scale command

kubectl scale deployments/kubernetes-bootcamp --replicas=4

To list your Deployments once again, use get deployments:

kubectl get deployments

The change was applied, and we have 4 instances of the application available. Next, let’s check if the number of Pods changed:

kubectl get pods -o wide

There are 4 Pods now, with different IP addresses. The change was registered in the Deployment events log. To check that, use the describe subcommand:

kubectl describe deployments/kubernetes-bootcamp

You can also view in the output of this command that there are 4 replicas now.

Scale Down

To scale down the Deployment to 2 replicas, run again the scale subcommand:

kubectl scale deployments/kubernetes-bootcamp --replicas=2

kubectl get deployments
kubectl get pods -o wide


Scaling overview

Performing a Rolling Update

for 

Updating an application

To update the image of the application to version 2, use the set image subcommand, followed by the deployment name and the new image version:

kubectl set image deployments/kubernetes-bootcamp kubernetes-bootcamp=jocatalin/kubernetes-bootcamp:v2
#deployment.apps/kubernetes-bootcamp image updated

The command notified the Deployment to use a different image for your app and initiated a rolling update. 

Verify an update

check that the app is running
kubectl describe services/kubernetes-bootcamp

#confirm the update by running the rollout status subcommand:

kubectl rollout status deployments/kubernetes-bootcamp


Let’s perform another update, and try to deploy an image tagged with v10:

kubectl set image deployments/kubernetes-bootcamp kubernetes-bootcamp=gcr.io/google-samples/kubernetes-bootcamp:v10

Use get deployments to see the status of the deployment:

kubectl get deployments

Notice that the output doesn't list the desired number of available Pods. Run the get pods subcommand to list all Pods:

kubectl get pods

Notice that some of the Pods have a status of ImagePullBackOff.

To get more insight into the problem, run the describe pods subcommand:
kubectl describe pods

In the Events section of the output for the affected Pods, notice that the v10 image version did not exist in the repository.

To roll back the deployment to your last working version, use the rollout undo subcommand:

kubectl rollout undo deployments/kubernetes-bootcamp 
#deployment.apps/kubernetes-bootcamp rolled back

Use the get pods subcommand to list the Pods again:

kubectl get pods

Four Pods are running. To check the image deployed on these Pods, use the describe pods subcommand:

kubectl describe pods


#clean up local cluster
kubectl delete deployments/kubernetes-bootcamp services/kubernetes-bootcamp

Troubleshooting with kubectl
  • kubectl get - list resources
    kubectl get pods # like docker ps  
  • kubectl describe - show detailed information about a resource
    kubectl describe pods # like docker inspect -it $container_name 
  • kubectl logs - print the logs from a container in a pod
    kubectl logs "$POD_NAME"    # like docker logs -it $container_name
  • kubectl exec - execute a command on a container in a pod
    kubectl exec -it $POD_NAME -- bash # like docker exec -it $container_name