Home>ASP source code>Other categories
Download

Shopping Cart Application with Microservices Approach

Building microservices application (Shopping Cart Application - Polyglot for services) using Kubernetes + Istio with its ecosystem parts.

Disclamation

  • Should have MINIKUBE_HOME environment variable in your machine, and value should point to C:users<your name>
  • Should run powershell script to create minikube machine in C: drive.
  • If it threw the exception that it couldn't find out minikube machine in Hyper-V so just simply delete everything in <user>/.minikube folder, but we could keep cache folder to avoid download everything from scratch, then runs it subsequently.

Table of contents

  • Technical Stack
  • Setup Local Kubernetes
  • Setup Istio
  • Setup Ambassador
  • Install and Work with Helm
  • Build Our Own Microservices
  • Available Microservices
  • Develop A New Service
  • Metrics Collection, Distributed Tracing, and Visualization
  • Tips and Tricks

Technical Stack

  • Hyper-V or VirtualBox
  • Docker
  • Kubernetes (minikube v0.25.2 for windows)
  • Istio
  • Ambassador
  • Helm
  • Weave Scope on Kubernetes
  • .NET Core SDK
  • NodeJS
  • Windows Powershell
  • xip or nip for access virtual hosts on your development web server from devices on your local network, like iPads, iPhones, and other computers.

Setup Local Kubernetes

  • Using minikube for Windows in this project, but you can use Mac or Linux version as well

  • Download the appropriate package of your minikube at https://github.com/kubernetes/minikube/releases (Used v0.25.2 for this project)

  • Install it into your machine (Windows 10 in this case)

  • After installed minikube, then run

Hyper-V

> minikube start --kubernetes-version="v1.9.0" --vm-driver=hyperv --hyperv-virtual-switch="minikube_switch" --cpus=4 --memory=4096 --v=999 --alsologtostderr

Then start with full option

> minikube start --extra-config=apiserver.Features.EnableSwaggerUI=true,apiserver.Authorization.Mode=RBAC,apiserver.Admission.PluginNames=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota --v=999 --alsologtostderr

VirtualBox v5.2.8

> minikube start --vm-driver="virtualbox" --kubernetes-version="v1.10.0" --cpus=4 --memory 4096 --extra-config=apiserver.authorization-mode=RBAC,apiserver.Features.EnableSwaggerUI=true,apiserver.Admission.PluginNames=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota --v=7 --alsologtostderr

Setup Istio

  • Download the appropriate package of Istio at https://github.com/istio/istio/releases

  • Upzip it into your disk, let say D:istio

  • cd into D:istio, then run

> kubectl create -f install/kubernetes/istio.yaml

or

> kubectl create -f install/kubernetes/istio-auth.yaml

Notes: set istiobinistioctl.exe to the PATH of the windows.

Setup Ambassador

  • If you're running in a cluster with RBAC enabled:
> kubectl apply -f https://getambassador.io/yaml/ambassador/ambassador-rbac.yaml
  • Without RBAC, you can use:
> kubectl apply -f https://getambassador.io/yaml/ambassador/ambassador-no-rbac.yaml
  • If you're going to use Ambassador, then run as following script
> cd k8s
> istioctl kube-inject -f istio-shopping-cart.yaml | kubectl apply -f -
> kubectl apply -f ambassador-service.yaml

Notes: for some reason, I couldn't run the no-rbac mode on my local development.

Dashboard

> minikube dashboard


> kubectl get svc -n istio-system

> export GATEWAY_URL=$(kubectl get po -l istio-ingress -n istio-system -o jsonpath='{.items[0].status.hostIP}'):$(kubectl get svc istio-ingress -n istio-system -o jsonpath='{.spec.ports[0].nodePort}')

> curl $GETWAY_URL

Install and Work with Helm (Kubernetes Package Manager)

> choco install kubernetes-helm

> cd <git repo>
> helm init
> helm repo update
> helm version
  • Install RabbitMq
> helm install --name my-rabbitmq --set rbacEnabled=false stable/rabbitmq

Now we can use amqp://my-rabbitmq.default.svc.cluster.local:5672 on Kubernetes Cluster, but what if we want to leverage it for the local development. The solution is port-forward it to our localhost as

> kubectl get pods | grep rabbitmq | awk '{print $1;}'
> kubectl port-forward <pod name just got> 15672

Or port-forward 5672 on Kubernetes (amqp protocol) to localhost:5672

> kubectl port-forward <pod name just got> 1234:5672

Now we have

> amqp://root:[email protected]:1234
  • Install Redis
> helm install --name my-redis stable/redis
  • References:
    • Stable Helm Template
    • How to install Helm
    • Issue: cannot access to 127.0.0.1

Build Our Own Microservices

  • Run
> minikube docker-env
  • Copy and Run
> @FOR /f "tokens=*" %i IN ('minikube docker-env') DO @%i

From now on, we can type docker images to list out all images in Kubernetes local node.

  • Build our microservices by running
> powershell -f build-all.ps1
  • Then if you want to just test it then run following commands
> cd k8s
> kubectl apply -f shopping-cart.yaml

  • In reality, we need to inject the sidecards into microservices as following
> cd k8s
> istioctl kube-inject -f shopping-cart.yaml | kubectl apply -f -

  • In Deployment

  • In each Pod

Available Microservices

  • Get host IP
> minikube ip
  • Get Ambassador port
> kubectl get svc ambassador -o jsonpath='{.spec.ports[0].nodePort}'

  • Finally, open browser with <IP>:<PORT>

  • Microservices

    • Catalog service: www.<IP>.xip.io:<PORT>/c/swagger/. For example, http://www.192.168.1.6.xip.io:32097/c/swagger/
    • Supplier service: www.<IP>.xip.io:<PORT>/s/
    • Security service: www.<IP>.xip.io:<PORT>/id/account/login or www.<IP>.xip.io:<PORT>/id/.well-known/openid-configuration
    • Email service: www.<IP>.xip.io:<PORT>/e/

Develop A New Service

  • Build the whole application using
> powershell -f build-all.ps1
  • Then run
> kubectl delete -f shopping-cart.yaml
  • And
> kubectl apply -f shopping-cart.yaml
  • Waiting a sec for Kubernetes to refresh.

Metrics Collection, Distributed Tracing, and Visualization

Setup Prometheus

> cd istioinstallkubernetesaddons
> kubectl apply -f prometheus.yaml

Setup Grafana

> cd istioinstallkubernetesaddons
> kubectl apply -f grafana.yaml

> kubectl -n istio-system port-forward $(kubectl -n istio-system get pod -l app=grafana -o jsonpath='{.items[0].metadata.name}') 3000:3000 &

> curl http://localhost:3000

Setup Service Graph

TODO

Setup Zipkin or Jaeger

TODO

Setup Weave Scope

  • Install and run it on the local
> kubectl apply -f "https://cloud.weave.works/k8s/scope.yaml?k8s-version=v1.9.0"
  • Then port-forward it out as following
> kubectl get -n weave pod --selector=weave-scope-component=app -o jsonpath='{.items..metadata.name}'
> kubectl port-forward -n <weave scope name> 4040
  • Go to http://localhost:4040

Tips and Tricks

  • Print out environment variables in one container
> kubectl get pods

> kubectl exec <pod name> env
  • Switch to another use-context

Let say we have a profile named minikube19, then just type the command as below

> kubectl config use-context minikube19

Switched to context "minikube19".

> minikube config set profile minikube19

Working with Ubuntu on Hyper-V

  • Install Ubuntu Server VM (ubuntu-18.04-live-server-amd64.iso) on Hyper-V (External Network)
  • ssh into that machine, then sudo apt-get install docker.io
  • Install Virtual Box (5.2) on Ubuntu VM
  • Install kubectl
> apt-get update && apt-get install -y apt-transport-https
> curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
> cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
> apt-get update
> apt-get install -y kubectl
  • Install minikube
> curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.27.0/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/
  • Alias command
> alias k='kubectl'
> alias mk='/usr/local/bin/minikube'
  • Then run following script
> minikube start --vm-driver="virtualbox" --kubernetes-version="v1.9.0" --cpus=4 --memory 4096 --extra-config=apiserver.authorization-mode=RBAC --v=7 --alsologtostderr
  • Make Hyper-V treats well with Ubuntu by enable nested vitualization, before of that we need to make Powershell run well
  • Make the internal Kubernetes Dashboard can be accessed by outside
> sysctl net.ipv4.ip_forward=1
> iptables -t nat -A PREROUTING -p tcp -d <VM IP> --dport 30000 -j DNAT --to-destination 192.168.99.100:30000
> iptables -t nat -A POSTROUTING -j MASQUERADE

> sudo iptables -t nat -v -x -n -L
> sudo dhclient eth0 -v

  • Install samba on ubuntu VM

  • socat on Ubuntu Server

> sudo apt-get install socat
> socat -v tcp-listen:30000,fork tcp:192.168.99.100:30000
  • ngrok on Ubuntu server
> wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip
> unzip ngrok-stable-linux-amd64.zip
> mk dashboard --url
> ./ngrok http 192.168.99.100:30000 -region ap
  • Open ubuntu VM port (if needed)
> sudo iptables -I INPUT -p tcp --dport 2375 -j ACCEPT
or
> sudo ufw allow 2375
then
> nc -l 2375
Expand
Additional Information
Related Applications
Recommended for You
Related Information All