https://www.henrydu.com/ Recent content on Henry Du Blog Hugo -- gohugo.io en-us Sat, 13 Jan 2024 00:00:00 +0000 https://www.henrydu.com/2024/01/13/netfilter-iptables-1/ Sat, 13 Jan 2024 00:00:00 +0000 https://www.henrydu.com/2024/01/13/netfilter-iptables-1/ Reading Notes: Netfilter and iptables Netfilter is firewalling subsystem in the Linux kernel as a modular[1]. For example, some fields are included in the sk_buf data structure only if the kernel is compiled with support for Netfilter feature. struct sk_buff is defined in linux/include/skbuff.h. There is a well defined document on the top of the file. It include the netfilter common header file #if IS_ENABLED(CONFIG_NF_CONNTRACK) #include <linux/netfilter/nf_conntrack_common.h>#endif Firewalling, essentially, hooks into certain places in the network stack code that packets always pass through when the packets or the kernel meet certain conditions. https://www.henrydu.com/2022/01/09/golang-inversion-of-control/ Sun, 09 Jan 2022 00:00:00 +0000 https://www.henrydu.com/2022/01/09/golang-inversion-of-control/ Golang Inversion of Control IoC Introduction Inversion of Control (IoC) is a programming design pattern which follows Dependency Inversion principle (DIP). It achieves decoupling the control logic and business logic. For instance, we normally will add control logic in the business logic, because it is very common in terms of human beings thinking logic. For example, we may add toggle-switch object when implement light bulb. We may also implement customized toggle-switch when making another electric device. https://www.henrydu.com/2022/01/05/golang-decorator-pattern/ Wed, 05 Jan 2022 00:00:00 +0000 https://www.henrydu.com/2022/01/05/golang-decorator-pattern/ Golang Decorator Pattern Decorator pattern is the one of twenty-three well-known design patter from Gang of Four design patterns. It is categorized in structural design pattern, along with Adapter pattern, Proxy pattern etc. The decorator pattern allows behavior to be added to an individual object, dynamically, without affecting the behavior of other objects from the same class. It tries to resolve the problem that single responsibility, following the Single Responsibility Principle, should be added to or removed from an object dynamically at run-time. https://www.henrydu.com/2021/12/01/cross-host-container-network-vxlan-mode/ Wed, 01 Dec 2021 00:00:00 +0000 https://www.henrydu.com/2021/12/01/cross-host-container-network-vxlan-mode/ Cross-host Container Networks - VXLAN mode VXLAN We have reviewed the UDP mode of cross-host container networks. We realized that there will be a performance issue because the IP packets have to be processed by the user space application. VXLAN allows us to handle the encapsulation/decapsulation in Linux kernel space. Virtual Extensible LAN a.k.a VXLAN (RFC7348) is supported by Linux kernel. It is able to encapsulate the IP packets by another layer and construct the overlay networks like the UDP tunnel does. https://www.henrydu.com/2021/11/28/cross-host-container-network-udp-mode/ Sun, 28 Nov 2021 00:00:00 +0000 https://www.henrydu.com/2021/11/28/cross-host-container-network-udp-mode/ Cross-host Container Networks - UDP mode In the previous article, we walked through containers network in one Linux host. However, by default Docker configuration, one container is unable to communicate with the other container in different host. In order to solve cross-host container communication problem, there are many solutions. We will go through some of them one by one. First and very famous one is Flannel CNI. Flannel CNI Flannel CNI project is the one of the projects developed by CoreOS. https://www.henrydu.com/2021/11/23/container-networks/ Tue, 23 Nov 2021 00:00:00 +0000 https://www.henrydu.com/2021/11/23/container-networks/ Container Networks Network Namespace Linux container network stack is isolated in its own network namespace. The network stack includes: Network Interface, Loopback Device, Routing Table and iptables chains/rules. The containerized process will use its own network stack to send and response the network request. If the running container wants to use the host network stack, the --net=host option will be used. Let’s start a Nginx container by sharing the host network. https://www.henrydu.com/2021/11/20/cgroups-container-resource-limitation/ Sat, 20 Nov 2021 00:00:00 +0000 https://www.henrydu.com/2021/11/20/cgroups-container-resource-limitation/ Container Resource Limitation Cgroups A container is running as a single process in a host. If multiple containers are running in the same host, there is a chance that one container will occupy the full CPU, while others are in the starvation situation. Therefore, it would be better to have a resource limitation, such that, one container will not fully share the host resources, such as CPU, memory etc. Linux Cgroups is the feature to limit resources for a process in Linux kernel. https://www.henrydu.com/2021/11/13/deep-dive-kubernetes-container-pid-isolation/ Sat, 13 Nov 2021 00:00:00 +0000 https://www.henrydu.com/2021/11/13/deep-dive-kubernetes-container-pid-isolation/ Deep Dive Kubernetes Notes: Container Process Isolation Process The binary executables are stored in the file system as a file. When the operating system starts to run one executable, it will load the file into the memory. For example, Linux executable has ELF format. In memory layout, it has a text area containing all the instructions. The instruction will be executed by CPU as an execution path. In the meantime, there may be files, I/O devices open and close associated with these executions. https://www.henrydu.com/2021/02/07/golang-interface-brief-look/ Sun, 07 Feb 2021 00:00:00 +0000 https://www.henrydu.com/2021/02/07/golang-interface-brief-look/ Golang Interface Source Code Brief Look Based on go v1.13, there are two struct definition for go interface: eface and iface. type eface struct { _type *_type data unsafe.Pointer } type iface struct { tab *itab data unsafe.Pointer } eface The eface represents the interface which does not have any method: interface{}. The eface has two fields: _type and data. data is actually a pointer to point to real data. _type is defined as follows: https://www.henrydu.com/2020/12/12/k8s-runtime/ Sat, 12 Dec 2020 00:00:00 +0000 https://www.henrydu.com/2020/12/12/k8s-runtime/ K8S Container Runtime Evolution With Docker In Kubernetes v1.20 release note, the major change is to deprecate dockershim, which means, K8S will never use Docker as container runtime. The Kubernetes community has written a blog post about this in detail. The docker runtime is just one component of Docker suite. Developers still use Docker to compile a docker image, and use docker hub to store docker images, as a docker image repository. https://www.henrydu.com/2020/12/03/cilium-service-lb-2/ Thu, 03 Dec 2020 00:00:00 +0000 https://www.henrydu.com/2020/12/03/cilium-service-lb-2/ Cilium’s service LB based on eBPF/XDP The first part introduced K8S service. Now, we can focus on Cilium’s service LB based on eBPF/XDP. Cilium agent Cilium service LB implements data path for all K8s service types via BPF. cilium-agent on each node watches kube-apiserver. cilium-agent observes the changes in K8S service. cilium-agent dynamically update BPF configuration according to changes in K8S service. As the diagram shown above, there are two components realized the service https://www.henrydu.com/2020/11/30/cilium-service-lb-1/ Mon, 30 Nov 2020 00:00:00 +0000 https://www.henrydu.com/2020/11/30/cilium-service-lb-1/ Cilium: K8S Service Load Balancing - Part 1 This blog is my reading note when I read K8S Service Load Balancing with BPF & XDP, presented by Daniel Borkmann and Martynas Pumputis in Linux Plumbers Conference. Kubernetes Networking Basic I have summarized kubernetes network feature when I introduced Flannel CNI. Kubenetes network is a flat network in the sense that each pod must be reachable by its IP address within a cluster. https://www.henrydu.com/2020/11/29/golang-add-subtract/ Sun, 29 Nov 2020 00:00:00 +0000 https://www.henrydu.com/2020/11/29/golang-add-subtract/ Add Subtract Coding Problem Problem Write a function addSubtract to meet the requirement that, it will alternately adds and subtracts carried parameters. For example: add_subtract(7) -> 7 add_subtract(1)(2)(3) -> 1 + 2 - 3 -> 0 add_subtract(-5)(10)(3)(9) -> -5 + 10 - 3 + 9 -> 11 Solution The idea is to have a addSubtract struct to keep tracking the last add/subtract result and the count of function calls. If the count is even number, then use subtract operation, otherwise, use add operation. https://www.henrydu.com/2020/11/21/kea-dhcp-webinar-05/ Sat, 21 Nov 2020 00:00:00 +0000 https://www.henrydu.com/2020/11/21/kea-dhcp-webinar-05/ Using KEA DHCP Webinar part 5 This blog is a study note of using Kea DHCP Webinar 05. It introduced Stork monitoring tool, logging and performance test tool, very briefly. Stork Monitoring Stork is a graphic dashboard for Kea DHCP server. It monitors Kea and Kea HA state. It alerts failures, fault conditions and other unwanted events. However, it is under active development. Stork can be installed to various platforms, including Ubuntu, Fedora and RedHat. https://www.henrydu.com/2020/11/16/k3s-cni-flannel/ Mon, 16 Nov 2020 00:00:00 +0000 https://www.henrydu.com/2020/11/16/k3s-cni-flannel/ K3S Supports Container Network Interface (CNI) and Flannel Introduction Kubernetes network provide the following features: Pods can communicate directly with all other pods on all nodes (no NAT). All nodes agents can communicate with other pods on all nodes (no NAT). The IP that a container sees itself as is the same IP that others see it as. This article explains how every pod is assigned an IP address managed by K3S. https://www.henrydu.com/2020/11/08/golang-decoupling/ Sun, 08 Nov 2020 00:00:00 +0000 https://www.henrydu.com/2020/11/08/golang-decoupling/ Golang Decoupling, Embedding and Exporting Decoupling Golang does not have a concept of object-oriented programming (OOP), including the concept of class or a class-based inheritance. Rather, Golang has its own way to interpret the regular object-oriented languages - decoupling. In traditional OOP design, one class contains member variables, either public or private. It also contains member functions, either public or private. In order to reuse the class member function, another class has to inherit the class. https://www.henrydu.com/2020/11/01/kea-dhcp-webinar-04/ Sun, 01 Nov 2020 00:00:00 +0000 https://www.henrydu.com/2020/11/01/kea-dhcp-webinar-04/ Using KEA DHCP Webinar part 4 This blog is a study note of using KEA DHCP webinar 04. This webinar introduced backend database and DHCP HA. For DB part, I only took very basic notes. The details of backend database can be available form the slides. Database backend support for Kea Kea DHCP server can store information in a database, including lease info, host address and prefixes, host options, host names and host classification. https://www.henrydu.com/2020/10/27/golang-channels/ Tue, 27 Oct 2020 00:00:00 +0000 https://www.henrydu.com/2020/10/27/golang-channels/ Golang Channel Patterns Golang channel allows us to transfer data structure between go-routine boundaries. It could be zero byte data struct{}{}, served as signaling purpose. In this use case, the channel is semantically a signaling, by which, one go-routine can send a signal to another go-routine, as a notification or an event. Signaling without data serves the main purpose of cancellation. It allows one go-routine to signal another go-routine to cancel the channel. https://www.henrydu.com/2020/10/26/golang-go-routines/ Mon, 26 Oct 2020 00:00:00 +0000 https://www.henrydu.com/2020/10/26/golang-go-routines/ Go Routine Run as a Thread? Whether or not a go-routine run as a machine thread is always a question for me, until I finally have a chance to inspect it by gdb. First, We could write a simple go code to have a go func(). package main import ( "fmt" "time" ) func main() { ch := make(chan string) go func() { fmt.Println(fmt.Sprintf("received the data '%v' from the channel", <-ch)) }() fmt. https://www.henrydu.com/2020/10/24/running-a-simple-operator/ Sat, 24 Oct 2020 00:00:00 +0000 https://www.henrydu.com/2020/10/24/running-a-simple-operator/ Running A Simple Operator in Minikube Introduction You may have heard of Kubernetes operator pattern. Maybe you have worked on one of projects that has Custom Resource Definition (CRD). Custom resource definitions are cool things once you define your custom resource on Kubernetes by using YAML. The Kubernetes API server component will implement CRUD API for you automatically. Then, you can use powerful Kubernetes clients, such as kubectl to interact with your own resources managed by Kubernetes. https://www.henrydu.com/2020/10/18/iptables-basic/ Sun, 18 Oct 2020 00:00:00 +0000 https://www.henrydu.com/2020/10/18/iptables-basic/ Basic iptables Introduction Kubernetes kube-proxy provides the important role to work with Service resource. kube-proxy will create and define iptables rules to manipulate the network traffic for the service resource. iptables is a Linux firewall. In this article, we will introduce very basic iptables concepts: tables, chains and targets. Tables are files are join similar actions. There are several build-in tables. Chains are the set of rules. Rules are statements about what to do with the network package. https://www.henrydu.com/2020/10/16/kea-dhcp-webinar-03/ Fri, 16 Oct 2020 00:00:00 +0000 https://www.henrydu.com/2020/10/16/kea-dhcp-webinar-03/ KEA DHCP part 3 This is the study notes of Using the Kea DHCP Server -Session 3 of 6. The DHCP reservation part will be added later on. DHCP Lease allocation When DHCP lease request is received, after Kea server successfully granted a lease, the following log is generated. 2020-10-15T20:35:36.137822+00:00 onprem-dhcp-6c98f7f9d9-lxd5d kea-dhcp4: INFO [kea-dhcp4.leases] DHCP4_LEASE_ALLOC [hwtype=1 10:7d:b9:2f:0d:eb], cid=[no info], tid=0x1: lease 172.17.10.1 has been allocated This informational message indicates that the server successfully granted a lease in response to client’s DHCPREQUEST message. https://www.henrydu.com/2020/10/05/dnstap-basic/ Mon, 05 Oct 2020 00:00:00 +0000 https://www.henrydu.com/2020/10/05/dnstap-basic/ DNSTap Basic Introduction DNSTap is a fast, flexible method for capturing and logging DNS traffic. Developed by Robert Edmonds at Farsight Security, Inc., and supported by multiple DNS implementations, dnstap uses libfstrm (a lightweight high-speed framing library, see https://github.com/farsightsec/fstrm) to send event payloads which are encoded using Protocol Buffers (libprotobuf-c, a mechanism for serializing structured data developed by Google, Inc.; see https://developers.google.com/protocol-buffers). Enable DNSTap for BIND To enable dnstap at BIND compile time, the fstrm and protobuf-c libraries must be available, and BIND must be configured with –enable-dnstap. https://www.henrydu.com/2020/10/05/kea-dhcp-webinar-02/ Mon, 05 Oct 2020 00:00:00 +0000 https://www.henrydu.com/2020/10/05/kea-dhcp-webinar-02/ KEA DHCP Part 2 This is the study notes from KEA DHCP Webinar Part 2. Some of contents are from ISC KEA document listed in Reference section. KEA DHCP KEA is modern DHCPv4 and DHCPv6 server. It is under open source MPL License. It provides JSON/REST API and modular design. It also provides high performance, which is able to have 1000 leases/seconds. KEA supports for DHCPv6 prefix delegation, dynamic reconfiguration, dynamic DNS updates, SQL database backend and PXE Boot support. https://www.henrydu.com/2020/09/15/kea-dhcp-webinar-01/ Fri, 18 Sep 2020 00:00:00 +0000 https://www.henrydu.com/2020/09/15/kea-dhcp-webinar-01/ Using the Kea DHCP Server 1 Introduction This is the study notes from ISC webinar DHCP is short for Dynamic Host Configuration Protocol. It is addressed in RFC 2131 for IPv4 and RFC 8415 for IPv6. DHCPv4 Protocol The DHCPv4 server or relay agents listen on port 67. The DHCPv4 clients listen on port 68 for messages from a server or relay agent. The communication between a client and a DHCP server follow the DORA procedures. https://www.henrydu.com/2020/09/13/golang-pass-map-value/ Sun, 13 Sep 2020 00:00:00 +0000 https://www.henrydu.com/2020/09/13/golang-pass-map-value/ Golang Pass Map Type Parameter Observation When I read this blog section “But maps and channels are references, right?”, I realize that the following code should print true, because Golang pass the map by value, not map address. package main import "fmt" func fn(m map[int]int) { m = make(map[int]int) } func main() { var m map[int]int fn(m) fmt.Println(m == nil) } Analysis This is running result. When the program starts run main function, the active stack frame in memory belongs to main. https://www.henrydu.com/2020/09/13/k8s-creates-pods/ Sun, 13 Sep 2020 00:00:00 +0000 https://www.henrydu.com/2020/09/13/k8s-creates-pods/ Kubernetes Creates Pods: Imperative vs Declarative Introduction Design and implementation microservices may have different approaches. During the design phase, we may need a quick proof of concept, or even multiple PoCs to compare. However, after we have a solid design, we may need to carefully plan how those microservices are working together in the cloud production. It demands us to specifically write Deployment, Service, ServiceAccount, ConfigMap etc. then, compile them to be a helm chart. https://www.henrydu.com/2020/09/12/docker-postgres-golang-db-ping/ Sat, 12 Sep 2020 00:00:00 +0000 https://www.henrydu.com/2020/09/12/docker-postgres-golang-db-ping/ Local Postgres Container Setting Up This article has been done last couple of years, but it is still useful for developers who are during transition from old school services to the docker container world. For example, we want to implement a service to access local Postgres DB. We can actually apt-get install, yum install or brew install a postgres DB service. However, since we don’t want our local OS has heavy load to run Postgres all the time, using Docker to have a Postgres is a better choice. https://www.henrydu.com/2020/09/11/minikube-docker-image-management/ Fri, 11 Sep 2020 00:00:00 +0000 https://www.henrydu.com/2020/09/11/minikube-docker-image-management/ Minikube manage docker images Minikube Minikube is a tool that make it easily to run Kubernetes locally. Installing minikube is very naturally for MacOS. After install minikube, we can check minikube status > minikube status minikube type: Control Plane host: Running kubelet: Running apiserver: Running kubeconfig: Configured We also can list default minikube pods > kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-66bff467f8-lnt67 1/1 Running 0 10d etcd-minikube 1/1 Running 0 10d kube-apiserver-minikube 1/1 Running 0 10d kube-controller-manager-minikube 1/1 Running 0 10d kube-proxy-pqg98 1/1 Running 0 10d kube-scheduler-minikube 1/1 Running 0 10d storage-provisioner 1/1 Running 0 10d Create a nginx pod Let’s create a nginx service quickly https://www.henrydu.com/2020/09/07/network-namespace/ Mon, 07 Sep 2020 00:00:00 +0000 https://www.henrydu.com/2020/09/07/network-namespace/ Linux Network Namespace This is study notes from Network Namespace Introduction Network namespace is using for container like docker to implement network isolation. When we create a container, we create a network namespace for it. So the container is not able to see host network such as routing table and arp table. The container has its own routing table and arp table in its namespace. Operation Commands To create a new network namespace, say red, using a command: https://www.henrydu.com/2020/09/05/new-start-with-hugo/ Sat, 05 Sep 2020 00:00:00 +0000 https://www.henrydu.com/2020/09/05/new-start-with-hugo/ New Start with Hugo My previous blog was hosted by HostGator. It is a traditional LAMP server with WordPress. Since I have started to use AWS as a daily basis, maintaining a static website by using AWS S3 bucket will be a good choice. AWS provided the detailed document to show how to configure a static website using a custom domain registered with Route 53. AWS also provided the document how to host a static website. https://www.henrydu.com/my-draft-post/ Thu, 01 Jan 1970 00:00:00 +0000 https://www.henrydu.com/my-draft-post/ once we defined a kubernetes service, kube-proxy is actually programe iptables, to make sure if something the service is resolvable via DNS (inside the K8S) service-name.my-namespace.svc.cluster.local https://www.henrydu.com/my-draft-post/ Thu, 01 Jan 1970 00:00:00 +0000 https://www.henrydu.com/my-draft-post/ QUIC blog: https://blog.cloudflare.com/the-road-to-quic/ Another blog: https://www.fastvue.co/fastvue/blog/googles-quic-protocols-security-and-reporting-implications/ lightweight pub/sub: NATS https://medium.com/capital-one-tech/lightweight-cloud-native-messaging-with-nats-ad730ca2becf https://www.henrydu.com/my-draft-post/ Thu, 01 Jan 1970 00:00:00 +0000 https://www.henrydu.com/my-draft-post/ What is SAML: https://www.varonis.com/blog/what-is-saml/ OKAT SAML: https://developer.okta.com/docs/concepts/saml/ Cloud Design Pattern: https://docs.microsoft.com/en-us/azure/architecture/patterns/cqrs https://www.henrydu.com/search/placeholder/ Mon, 01 Jan 0001 00:00:00 +0000 https://www.henrydu.com/search/placeholder/ https://www.henrydu.com/top/about/ Mon, 01 Jan 0001 00:00:00 +0000 https://www.henrydu.com/top/about/ Henry Du is a software developer as well as network security engineer. He has a solid experience in the network and security industry for more than 10 years. // Work is always interesting type Work interface { // Coding like art Coding() string // Networking is fundamental Networking() string // CyberSecurity is critical CyberSecurity() string // CloudComputing is future CloudComputing() string } // Life is beautiful type Life interface { ListenMusic() string WatchMovies() string ReadBooks() string TakePhotos() string Sports() string Travel() string Blog() string } type henry struct { livingAt string workedFor string jobTitle string graduatedFrom string degree string cert string } func NewHenry() Life { return &henry{ livingAt: "Vancouver, Canada", workedFor: "Aviatrix", jobTitle: "Sr.