


Tldr; An attempt to help you to better understand the famous **etcd** by giving an introduction on overall architecture and algorithms



##  🛸 What is etcd ?

by definition etcd is a distributed, reliable key-value (KV) store for a distributed system. so simply etcd is just a consistent, fault-tolerant database used by modern tools like [kubernetes](https://kubernetes.io) to store (mainly) a **configuration** as a key-value structure (think about it a like JSON file). And as we are using etcd to store configuration the main design is to reliably store infrequently updated data and provide reliable watch queries.

The name comes from the GNU/Linux convention used to name configuration files “/etc” and the “d” stands for distributed.Etcd was created by the same team responsible for designing CoreOS Container Linux. In December 2018, the team donated etcd to the Cloud Native Computing Foundation (CNCF)

###  📦 Components

![](media/etcd-components.png)

The etcd is a toolchain with multiple components:

- **client**: etcd official Go client for etcd API (HTTP/GRPC) .

- **Balancer**: etcd client load balancer that implements retry and failover mechanism. etcd client should automatically balance loads between multiple endpoints.

- **Endpoints**: A list of etcd server endpoints that clients can connect to.



##  💡 How etcd works

etcd is based on an algorithm called Raft Consensus Algorithm ( an enhancement of consensus in Paxos algorithm ) to provide consistency and as etcd needs to be highly available it's run on cluster mode (multiple nodes); consensus stands for multiple distributed machines agree at same information (by-elections) 

![](media/raft-ss.png)

Let's say a client sends a message to the server and the server responds with a reply in the case of a single server there is no problem because there is only one server that will take care of the decision


![](media/raft-ms.png)

but in multiple servers  case (distributed system) the server response is related to:

- Type of system:

  - Symmetric: any server can respond to the request

  - ASymmetric: only the **elected** leader server can respond to the client

- System components:

  - Cluster: a group of machines (VMs, hardwares...)

  - Leader: The chosen machine by the cluster to be Leader; The leader ensures all the write operations and does log replication (for read operations any member can serve).

  - Elections: If a leader dies (didn't send a heartbeat after a configurable timeout) or fails to respond, the remaining nodes in the cluster elect a new leader.


![](media/raft-lr.jpg)

As the leader is the responsible of handling the write operations .. the write operations in Raft world is a log driven means the leader write a record in Raft log entry and broadcast it to all followers (consistency) once the Quorum (see next paragraph) acknowledged writing the change the mark the transaction as committed.

Another interesting concept ensuring etcd fault-tolerance is how many faulty nodes are tolerable in a given etcd cluster.. for that etcd uses Quorum.

Quorum is defined as __(n+1)/2__, indicating that more than half of the nodes in the cluster constitute a quorum. for example a 3 nodes cluster, etcd still runs as long as any 2 nodes are available.

Similarly, in a 5 nodes cluster, etcd still runs as long as any three nodes are available...


![](media/quorom.jpg)


## 🏗️ Design & Data model

From physical view etcd stores the physical data as key-value pairs in a persistent b+tree and also keeps a secondary in-memory btree (to speedup indexing). Each revision (see next paragraph) of the store’s state only contains the delta from its previous revision to be efficient. A single revision may correspond to multiple keys in the tree. see here for more details [Data model](https://etcd.io/docs/v3.5/learning/data_model/). 
The datastore is built on top of BoltDB, or more specifically, BBoltDB, a fork of BoltDB (a Go key-value store) and it's also maintained by the etcd team.

From logical perspective the store is a flat binary key space and the key space has a lexically sorted index on byte string keys so range queries are inexpensive.

Besides this key space maintains multiple revisions aka using MultiVersion Concurrency Control (MCCV) in order to handle concurrent operations. 

Example of revision in etcd using `etcdctl`

```sh
$ etcdctl put foo bar         # revision = 2
OK
$ etcdctl put foo1 bar1       # revision = 3
OK
$ etcdctl put foo bar_new     # revision = 4
OK
$ etcdctl put foo1 bar1_new   # revision = 5
OK

```

Alongside revision etcd uses a term to represent the term of the leader. When the leader changes, the term value increases by 1.


![](media/etcd-term.png)



## ⚡ API

Honestly, etcd has a complex internal architecture but exposes a very nice and intuitive:

![](media/etcd-api.png)


- `Put`: operation to write data to the cluster

- `Delete`: operation to delete data from the cluster

- `Get` : operation to retrieve data from the cluster with support to queries

- `Watch` : a real time watch mechanism to subscribe to the data updates in the cluster.

- `Transactions` : a simple transaction mechanism to perform at set operations (not in an atomic way)

- `Leases` : The Leases API is a design pattern used a within distributed system, think about it like a contract that gives its holder specified rights to some resource for a limited period, in etcd case is required to detect whether a node is alive in a distributed system.

![](media/lease-etcd.png)

Here a 10sec lease is created if no operations are done during this time it will automatically be cleared. After we bind PUTs operations to the leases so that etcd automatically clears key1 and key2 when the lease dies. for keeping a lease you need to call the `lease.KeepAlive()` method. The lease is used widely within K8s and the most useful use in my opinion is service discovery where the process (backends) register address to etcd when the process is launched so the API Gateway read from etcd when a failover occurs, and the new process address is registered with etcd and API Gateway can have the new ones.

![](media/sd-etcd.png)


## 🔌 Install

You can install a pre-build version of etcd like I'm using `yay` but use whatever you want:

```sh
yay -S etcd
```

or you can use docker version

```sh
docker run -d -v /usr/share/ca-certificates/:/etc/ssl/certs -p 4001:4001 -p 2380:2380 -p 2379:2379 \
 --name etcd quay.io/coreos/etcd:v2.3.8
```

After install, you can launch the daemon like 

```sh
etcd
```

![](media/etcd-screen1.png)


To play with it we will use the client CLI `etcdctl`

![](media/etcd-screen2.png)



Cheers




Just another blog post to demystify the famous etcd.

Breaking down etcd


Back to blogging again I'm learning golang and so I decided to create a simple CI platform.

## 🛸 What is Hermes ?

Hermes is the winged herald and messenger of the Olympian gods. In addition, he is also a divine trickster, and the god of roads, flocks, commerce, and thieves. ... Hermes was the only Olympian capable of crossing the border between the living and the dead.

Ohh nope not that ...

Hermes **CI** (CI stands for Continuous integration) is an open-source continuous integration platform mainly written in Go.

The main idea is to clone the core feature of [TravisCI](https://travis-ci.org/) means to run a pipeline in a code repository and save logs and build status whenever a developer performs a registered event (push, PR, merge ...)

## 🚦Continuous integration (Back to school)

> Continuous integration is a DevOps software development practice where developers regularly merge their code changes into a central repository, after which automated builds and tests are run — AWS

## Breaking down the solution:

![](media/hermes.png)

So the idea is to create a process of transit of the code from developers' machines to production servers automatically. And as `git` is become the defacto standard of version control.I will be using Github webhooks to capture the developer's events on the code.

### ⚙ Config file:

It's a simple config file to tell the backend the type of job to schedule. This PoC support only Dockerized application ( app with docker file)

> Hermes.yaml

```yaml
name: My super pipeline
version: 1.0.0
schema: docker
```

> Dockerfile 🐳

A classic `Dockerfile` with instructions to be executed by `buildh`. in this example I used a very simple one

```docker
FROM yauritux/busybox-curl

RUN echo "Running build"
RUN echo "We don't need to build anything"
RUN echo "Getting Ouarzazate weather"
RUN echo "weather is goood"
```

### 💅 Frontend :

Simple nuxt.js application allow the user to authenticate using Github's OAuth and get the list of public repositories.

```
├── README.md
├── commitlint.config.js
├── components
│   ├── NuxtLogo.vue
│   └── Tutorial.vue
├── jest.config.js
├── layouts
│   └── default.vue
├── middleware
│   └── authenticated.ts
├── nuxt.config.js
├── package.json
├── pages
│   ├── index.vue
│   ├── login.vue
│   └── repos
│       ├── _id
│       │   ├── index.vue
│       │   └── job
│       │       └── _job.vue
│       └── index.vue
├── static
│   └── favicon.ico
├── store
│   ├── README.md
│   ├── index.md
│   └── index.ts
├── stylelint.config.js
├── test
│   └── NuxtLogo.spec.js
├── tsconfig.json
├── types
│   └── index.ts
└── yarn.lock

```

![](media/hermes-f.jpeg)

The magic happens when the user chooses a repository and configures it with Hermes CI (basically creating a webhooks on the repository's end (Github) to listen to Push, Pull request event)

```js
    createHook() {
      try {
        this.$axios.post(
          `${process.env.gitHubUrl}repos/${this?.user?.login}/${this?.id}/hooks`,
          {
            name: 'web',
            active: true,
            events: ['push', 'pull_request'],
            config: {
              url: `http://hermes.soubai.me/github/${this?.repo?.id}`,
              content_type: 'json',
              insecure_ssl: '0',
              digest: 'Hermes',
            },
          },
          {
            headers: {
              Accept: 'application/vnd.github.v3+json',
            },
          }
        )
      } catch (error) {
        console.log(error)
      }
    },
```

Additionally, the frontend allows the user to track the build status and read the logs (fetching from the backend)

![](media/hermes-log.jpeg)

### 🦀 Backend

```
├── LICENSE
├── Makefile
├── go.mod
├── go.sum
├── main.go
├── models
│   └── Job.go
├── readme.md
├── server.service
└── tmp
```

A Golang HTTP server application to handle the request sent by Github whenever a registered event is triggered by user actions ex. `push`, `pull_request`. The backend uses the payload as context to create a job and schedule it in a queue message. Besides that, the backend saves status, logs, and jobs in MongoDB so we can read asynchronous operations in the frontend later.

> http server

```go

	router.HandleFunc("/github/{id}", handleGitHubWebhook).Methods("POST")
	router.HandleFunc("/github/{id}", GetJobs).Methods("GET")
	router.HandleFunc("/jobs/{id}", GetJob).Methods("GET")

	err := http.ListenAndServe(":"+os.Getenv("PORT"), corsOpts.Handler(router))
	if err != nil {
		log.Fatal(err)
	}

```

I use [Async](https://github.com/hibiken/asynq) as a message queue that uses Redis internally as a database in memory; The main goal of using Async is the classic use case of a message queue: optimize job requests load and decoupling a backend APIs part from the job Runner.

```go
func JobProcessingTask(id string, body Payload) (*asynq.Task, error) {
    payload, err := json.Marshal(JobPayload{ID: id, Body: body})
    if err != nil {
        return nil, err
    }
    return asynq.NewTask(TypeJobProcessing, payload), nil
}
```

### 🏃 Runner

```
├── Makefile
├── go.mod
├── go.sum
├── install.sh
├── main.go
├── models
│   └── Job.go
├── readme.md
└── tmp
```

The runner is an application that works with pipelines. A pipeline is a list of instructions/commands supposed to be executed by this runner as a single isolated process. In our case, the job is the pipeline defined in a `Dockerfile` inside the user's Github repository.

So basically runner's role is to :

1. Pull the user's repository code from Github
2. Run Dockerfile using [buildah](https://buildah.io/)
3. Catch build logs
4. Save everything in `MongoDB` database

```go
ctx := context.Background()
cmd = exec.CommandContext(ctx, "buildah", "bud", "--log-level", "info", "-t", strings.ToLower(body.Repository.FullName), path)
stdout, err := cmd.StdoutPipe()
cmd.Stderr = cmd.Stdout

```

## 📋 Sum up

HermesCI is a PoC of how we can make a CI system at home and gives me a great way to learn about all those concepts. you can find the code publicly here [buildah](https://github.com/soub4i/hermes-ci)

Cheers


Back to blogging again I'm learning golang and so I decided to create a simple CI platform.

Look mom no TravisCI


# Context

Web development is based on 3 principle languages HTML,CSS and Javascript those three ingredients are the backbone of every website the problems is we moved from a simple website to show up some information on a simple web page (web 1.0: Read-only) to more and more complex use case like an interactive webpage (web 2.0: Read-write) or more intelligent web (web 3.0: read-write + XR)…

More the complexity of the web application is increasing more the tooling involved in the creation and maintaining is heavy … this fact drives us to a reality that the only HTML,CSS, and Javascript are not enough anymore and we should use React,PUG,SCSS,Typescript, NPM packages …

More and more tools and this is another problem from **Developer experience** (browser compatibility and data exportability) and **optimization** (big bundle files) perspective … but hopefully we have bundlers like _Parcel_,_Rollup_,_Snowpack_ and **Webpack**

In today’s blog post we will see what is webpack and how it works and finally, we will end up by creating a simple plugin.

# Webpack

We said that the problem with a complex and modern web application is multiple module dependencies. Webpack as a module bundler has one important job is to take multiples dependencies (for example multiple js files) and combine that into one single file (or multiple chunks files).

To handle those different types of dependencies webpack uses **Loaders**; A Loader is a way to process files that are not javascript (we assumed that webpack is created to handle js files) for example we can configure webpack (via `webpack.config.js` file) to handle SVG,CSS,SCSS …

Some we need more complex/advanced treatment for our assets besides loading them here webpack uses **plugins** to achieve this routine. We can control webpack compilation with plugins by tapping directly to bundler lifecycle for example we can use a plugin to minify our CSS code or maybe apply some auto-prefixing …

Like loaders plugins can be added via `npm` and should be configurated using `webpack.config.js`

# Let’s code

Webpack plugin is just a javascript class with _apply_ method this method take `compiler` instance as a parameter so we can use `compiler.hooks.*` to register our custom logic based on a particular webpack compiler event (here we listen to `done`) so our plugin will be something like:

```js
// ./plugin/super-plugin.js
class SuperPlugin {
  apply(compiler) {
    compiler.hooks.done.tap('Super Plugin', (stats) => {
      console.log('Hello World! from Super Plugin');
    });
}
module.exports = SuperPlugin
```

and we can add it to our `webpack.config.js` file like this:

```js
const SuperPlugin = require("./plugin/super-plugin.js");
// ...
plugins: [new SuperPlugin()];
```

And we can add options to our plugin using the class _Constructor_

```js
// ./plugin/super-plugin.js
class SuperPlugin {
  constructor(options){
    this.options = options
  }
  apply(compiler) {
    compiler.hooks.done.tap('Super Plugin', (stats) => {
      console.log('Hello World! from Super Plugin');
      console.log('Options:', this.options);
    });
}
module.exports = SuperPlugin
```

you can try plugin by running package.json _dev_ script: `yarn dev`

```json
{
  "scripts": {
    "dev": "webpack"
  },

```

let's code something useful a plugin that give us the size of the output file :

```js
// ./plugin/super-plugin.js

const fs = require("fs");

class SuperPlugin {
  constructor(options){
    this.options = options
  }
  apply(compiler) {
    compiler.hooks.done.tap('Super Plugin', (stats) => {
         const { path, filename } = stats.compilation.options.output;
         const stats = fs.statSync(`${path}/${filename}`)
         const size = stats.size / (1024*1024);
         console.log(`Your output size is : ${size} MB`)

    });
}
module.exports = SuperPlugin
```

Cheers


In this blog post I will teach you how you can easily build a webpack plugin ...

Build your own webpack plugin


# Definition

Let's build a quine using javascript. but wait a minute what is QUINE ???

A Quine the name came from the book Gödel, Escher, Bach in honor of philosopher Willard Van Orman Quine (1908–2000) by Douglas Hofstadter and it significate a computer program that takes no input and can print its own source code. These programs called "self-replicating programs" or "self-reproducing programs".

The behavior of this program it's can be used to reproduce new programs in a given ecosystem think about a Worm (computer virus) that can reproduce itself through network devices.

> Imagine combining quining with AI :-o

# Rules

Mainly quines problems (as I know) are used as challenge or fun in computer science and if you want to make a quine in any programming language you need to fit 3 rules:

- Quine can't get any input or `args`

- You most achieve quining without using IO (read from the disk) so something like this isn't allowed

```js
console.log(require('fs').readFileSync('./index.js',{encoding:'utf8', flag:'r'}))
```

- You quine need to have at least 1 character because technically an empty program is a quine that print it's own source code _YEY CRAZY_

# Advanced Quines 
What we described here is a "vanilla" Quine but we have more advanced ones, grab a set:

### PolyQuine:

![](media/1_EVLTR5kscFOBdAxdEQ5nyA.png)

is a program that it's valid in more than one programing language at the same time and in each one, it's printing its source code.


### Ouroboros quine

![](media/langs.png)

It's `ruby` program that's =(create)=> `rust` program that's =(create)=> `scala` program ... and after 128 languages it's create a the **original** `ruby` program. This beautiful beast is created by japanase programmer Yusuke Endoh (ruby core language contributor); Check this [repo](https://github.com/mame/quine-relay) for more information.

# Hacking time
As I use javascript I'll try to build one using javascript basically I need to use a template literals and stringify the main function and put it as variable in the string template and what's makes my task relatively easy is all javascript functions have a `.toString()` method that is fired automatically when you try to treat a function as a string (print, concat ...); plus for a simplicity reason I will use a IIFE.

```js
(function q(){
  return '('+ q.toString() +')()'
})()
```

![](media/quine.png)

and for the sake of geekiness 

```js
$=_=> `$=${$};$()`;$()
```

![](media/quine_geeky.png)





Let's build a quine using javascript. but wait a minute what is QUINE ??? ...

Holy moly! Quines in javascript


# definitions

The first time I was asked "Define backoff algorithm" I mumbled WTH is Backoff algorithm I wasn't able to find anything in the words to gives me a lead to a generic definition so I can keep saving my Ego. I answered I don't know after that I searched and I got a lot of definitions but the clearest to me is: _Binary exponential backoff or truncated binary exponential backoff refers to an algorithm used to space out repeated retransmissions of the same block of data, often to avoid network congestion._

In simple words is an algorithm (sequence of well-defined, computer-implementable instructions) that makes sure to retry a failed action (fetch data from a server) until it succeeds without flooding the system with retries. And this achieved by putting waitings periods between retries to give a target time to recover.

This algorithm is widely used to resolve network problems such as collision, network congestion...

Retrying a failed fetch call to a server is the most easier implementation of this algorithm highly used in Networking such when the mobile network is saturated and your device is trying to connect to the endpoint (station) using this algorithm implementation

# Binary exponential backoff

The backoff algorithm is very good at solving this kind of problem but the challenge is how we can define a good/efficient "wait time" between retries because sending requests too soon can overwhelm our server and puts more loads. So here the standard is to use a Binary exponential backoff is a backoff strategy that waits exponentially longer for subsequent retries; For example, the first retry waits 1s and the subsequent ones double the previous values exponentially 1s, 2s, 4s, 8s, 16s, 32s… and that can allow our program to start with small waiting values and reach a big value quickly.

# Implementation

In this article, I will try to implement this algorithm using javascript to retry a failed fetch request.

there is an important thing to highlight every time you choose backoff strategy: **Make sure to put a limit to the number of retries** so that your program can throw error don't build a "GOD program"

Let's hack

let's say we want to retry our call request until it's successful or reach the value of the max attempts (zero because we going to do that **recursively**)

```js
// let's create a wait function that takes a duration

const wait = (duration) => new Promise((res) => setTimeout(res, duration));

// retry is a function that takes  maxAttempts (number of allow retries) , a callback (function to repeat) and a delay (time in ms)

const retry = (maxAttempts, cb, delay = 500) =>
  cb().catch((err) =>
    maxAttempts > 1
      ? wait(delay).then(() => retry(maxAttempts - 1, cb, delay * 2)) // *2 is the exponential rate
      : Promise.reject(err)
  );
```

And to finally

```js
// a fetch api from a very busy server

const call = fetch("https://bac2021.men.gov.ma/undifined-endpoint");

// RUN retry with call and  10 max retries and 500ms as starting wait time &&  implicitly pass the result of the promise to to the console.log

retry(10, call).then(console.log()).catch(console.log());
```

# Conclusion

Using the retry strategy is a very good choice in the sensitive systems and gives a better user experience to your applications however make sure you define a full stop point to your programs. Cheers!


The first time I was asked 'Define backoff algorithm' I mumbled WTH is Backoff algorithm...

Backoff Algorithm implementation in javascript


In this blog post I will try to demystify the web component concept for you and we'll create a simple web component as a demo

## What is Component (React as an example) ?

Let's take react as an example, react is a great frontend library focused on building UI (user interfaces) quickly and efficiently .

To build UI react (and other javascript frontend frameworks) use the components. And here component stands for the building block part for every UI; for example we have a dashboard page with a navbar, sidebar, chart... every part of UI elements is a component moreover the page is one big component includes all other ones and act as a container.

![](media/react-component-tree.png)

In react to create a component we have two ways :
1. function-based component:  where the component is a function 
2. class-based component: where the component  is a class extending `React.Component`

each one has pros and cons (React team is pushing to use functional components in the newest versions)

So what are web components and why we need them if we already have javascript frameworks?

## Web Component:

Web components are a set of **web platform APIs** that allows you to create new custom, reusable, encapsulated HTML tags to use in web pages and web apps.

In easy words we don't need javascript framework to create components we can use vanilla javascript to define our custom HTML elements while maintaining the encapsulation and reusability.

### Web components specifications:

According to [MDN](https://developer.mozilla.org/en-US/docs/Web/Web_Components) web components are: 

 > Suite of different technologies allowing you to create reusable custom elements — with their functionality encapsulated away from the rest of your code — and utilize them in your web apps.

those set of techos are the specifications to define valid web components :

 1. **Custom Elements**: The Custom Elements specification lays the foundation for designing and using new types of DOM elements.
 ![](media/unnamed-2-.png)
>Allow you to define custom elements and their behavior
       
 3. **Shadow DOM**: The shadow DOM specification defines how to use encapsulated style and markup in web components.
 ![](media/unnamed-3-.png)
> Allow you to attach an encapsulated "shadow" DOM tree to an element
       
 4.  **ES Modules**: The ES Modules specification defines the inclusion and    reuse of JS documents in a standards-based,
    modular, performant way.
       
![](media/pasted-image-0.png)
> ECMAScript standard for working with modules. While Node.js has been using the CommonJS standard for years, the browser never had a module system
       
 4.  **HTML Template**: The HTML template element specification defines how to declare fragments of markup that go unused on page load, but can be instantiated later on at runtime.
    
 ![](media/unnamed-1-.png)
> The `<template>` and `<slot>` elements enable you to write markup templates that are not displayed in the rendered page. These can then be reused multiple times as the basis of a custom element's structure.

### Browser Compatibility
An important thing that made web components hypes recently is the big adoption/compatibility with the modern browsers so we can use them without polyfills nor a bundlers 

 ![](media/bc.png)

## Demo
I'll create  a web component to load Rick & Morty charcters from rest API

```js
// index.js
class RMCharacter extends HTMLElement {

    // make the custom attribute observable with attributeChangedCallback
    static get observedAttributes() { return ['name']; }

    
    constructor() {
        super()
        // attache the shadow dome to our web component to ensure the style encapsulation
        this.shadowDOM = this.attachShadow({ mode: 'open' });
        this.shadowDOM.innerHTML = `
        <style>
            h3 {
            color : red
        }
        </style>
        `
    }

    // lifecycle method called when the web component is mounted
    async connectedCallback() {

        // call the Rick & Morty api to get the character by name
        let data = await fetch("https://rickandmortyapi.com/api/character/?name=" + this.name)
        let { results } = await data.json()
        // create html elements div, img, h3
        const warpper = document.createElement("div")
        const img = document.createElement("img")
        const name = document.createElement("h3")
        // set values to our html elements
        img.setAttribute("src", results[0].image)
        name.innerText = results[0].name
        warpper.appendChild(name)
        warpper.appendChild(img)
        // add everything to the root shadowDOM
        this.shadowDOM.appendChild(warpper)
    }

    // lifecycle method to catch changes of attributes

    attributeChangedCallback(attributeName, oldValue, newValue) {
        if (attributeName === "name" && oldValue !== newValue) {

            this.name = newValue

        }
    }
}


// define our web component to the browser
window.customElements.define('rm-character', RMCharacter)
```

```html
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
    <style>
        h3 {
            color: green
        }
    </style>
</head>
<body>
    <h3>My</h3>
    <rm-character name="Rick Sanchez"></rm-character>
    <rm-character name="morty"></rm-character>
    <script type="module" src="index.js"></script>
</body>
</html>
```

And the result is something like this :

![](media/sreenshot-demo.png)

As you can see we create a custom, reusable, encapsulated HTML element easily with web component specifications and this element can be re-used with any kind of frontend technologies (vanilla, jquery, javascript framework..) and within the same project or cross-projects 

## What's next :

Check this resources and tools :

- Polymer (Backed by Google)
- Stencil (Backed by Ionic)
- Skate.js (Works with other frameworks )
- Hybrids (UI lin for create WC)



Hello in this blog post I will try to demystify the web component concept for you

What's the heck is web components

Imagine you are a simple dude frontend developer minding his `flex` and `React.Component` stuff and one day your boss tells you to add a brain to you fat ass frontend app and make it smarter .. I know you will quit the job but if you are broke as fuck && you like to challenge yourself like me you will try to learn how to do this.

Well in this blog post I will give you simple steps to put machine learning into as frontend developer that has nothing to do with AI or ML before.



# 1. Definitions

*Definitions are almost mine as far as I understand the concept please note that I'm not a Data scientist* 

**AI (Artificial intelligence)** is the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.

**ML (Machine learning)** is the scientific study of algorithms and statistical models that computer systems use to effectively perform a specific task. AI (Artificial intelligence) is the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.

ML (Machine learning) is the scientific study of algorithms and statistical models that computer systems use to effectively perform a specific task.

### 1.2 Types of ML

![](media/0_-068ud_-o3ajwq_z.jpg)

1. Supervised learning: The machine learning algorithm is trained on labeled data. Even though the data needs to be labeled accurately for this method to work, supervised learning is extremely powerful when used in the right circumstances.
2. Unsupervised machine learning holds the advantage of being able to work with unlabeled data. This means that human labor is not required to make the dataset machine-readable, allowing much larger datasets to be worked on by the program.
3. Reinforcement learning directly takes inspiration from how human beings learn from data in their lives. It features an algorithm that improves upon itself and learns from new situations using a trial-and-error method. Favorable outputs are encouraged or ‘reinforced’, and non-favorable outputs are discouraged or ‘punished’.

### 1.3 Model

![](media/images.jpeg)

 A machine learning model (basically a neural network) can be a mathematical representation of a real-world process. To generate a machine learning model you will need to provide training data (dataset) to a machine-learning algorithm to learn from.

### 1.4 Traditional app paradigm vs ML app

![](media/tvsml.png)

In the traditional app we feed algorithms to our business logic layer to get answers (processed data) but in machine learning app we feed our model with data (answers) and we get a model (algorithms)

## 2. Tensorflow & Tensorflow.js

![](media/tf.png)

**TensorFlow** is an end-to-end open-source platform for machine learning. It has a comprehensive, flexible ecosystem of tools, libraries, and community resources that lets researchers push the state-of-the-art in ML, and developers easily build and deploy ML-powered applications.

**Tensorflow.js** (previously Deeplearn.js) the library by Google is GPU accelerated via WebGL API and used for predictions by using pre-trained models in inference mode in the browser but also for the training mode itself. It mirrors the API of the popular TensorFlow library.

### 2.1 Where Tensorflow run

Basiclly everywhere :

1. Desktop running Windows, macOS or Linux (TensorFlow)
1. Cloud as a web service (Google cloud)
1. Mobile devices like iOS and Android (TensorFlow Lite)
1. Browsers (Tensorflow.js)


### 2.1 Installation of Tensorflow.js

You can use it directly via CDN 

```html
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@1.0.0/dist/tf.min.js"></script>
```

OR using package manager : 

`
npm install @tensorflow/tfjs
`

for server side (node.js):

Option 1: Install TensorFlow.js with native C++ bindings.

`
npm install @tensorflow/tfjs-node
`

Option 2: Linux Only) If your system has a NVIDIA® GPU with CUDA support, use the GPU package even for higher performance.

`
npm install @tensorflow/tfjs-node-gpu
`

## 3. Case Study 1: Linear regression (Hello world)

![](media/lr.gif)

I will try with in this use case to train my model on the browser using tensorflow.js using a small dataset (just to demonstrate) to generate that will try to predict the price of a houses in Barcelona based on number of rooms 

```html
<html>
  <head>
    <title>Hello TensorFlow</title>
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@0.6.1"></script>
    <script>
      const nr_epochs = 248; // higher=better but slower
      var tfinterface;
      const model = tf.sequential();

      function initTF() {
        const xs = tf.tensor2d([3, 3, 4, 4, 5]);
        const ys = tf.tensor2d([131000, 125000, 235000, 265000, 535000]);

        // 2. ML MODEL - linear regression model
        model.add(tf.layers.dense({ units: 1, inputShape: [1] }));
        // Prep for training
        model.compile({ loss: "meanSquaredError", optimizer: "sgd" });

        // train -- the higher the number the more accurate you'll get (but longer run time)
        tfinterface = model.fit(xs, ys, { epochs: nr_epochs });
      }

      // 3. WRAPPER AROUND PREDICT
      function predict(n) {
        return tfinterface.then(() => {
          return model.predict(tf.tensor2d([n], [1, 1]));
        });
      }

      // Helper for form
      function formpredict(v, r) {
        predict(v).then(function(res) {
          //   alert(res.get([0]));
          r.innerHTML = res.get([0]);
        });
      }
    </script>
  </head>

  <body onLoad="initTF();">
    <p></p>
    <div id="res">Wait</div>
    <p></p>
    <form
      name="iForm"
      onSubmit="formpredict(this.val.value,document.getElementById('res')); return false;"
      )
    >
      <script>
        document.getElementById("res").innerHTML = "&nbsp;";
      </script>
      Input Number: <input name="val" /><input type="submit" />
    </form>
  </body>
</html>
```

Here I trained the model and add an event listener to predict function on submitting the form 

[Code Source](https://codesandbox.io/s/determined-allen-2v1i7?file=/index.html)

## 4. Case Study 2: Coriander and Parsley classifier

In this case, I will try to create a model and use it to make predictions (identify coriander and parsley) I will not train the model in the browser like the first case for performance reasons (I used google collab [Here](https://colab.research.google.com/drive/135Oblj6v_Y2uAJOf3quw9y3F38idJLIZ) ) with this [dataset](https://github.com/alilakrakbi/Coriander-vs-Parsley)  Kudos to [Ali Lakrakbi](https://twitter.com/alilakrakbi), The output of trained model is a Keras model (Model.p5) I used [tfjs-converter](https://github.com/tensorflow/tfjs/tree/master/tfjs-converter) to my model to Tensorflow.js compliant Model.

Finally, I just wrote an "Exploitation" code in javascript to use a pre-trained model to predict the picture if it's coriander or parsley

You can found the example [here](https://codesandbox.io/s/proud-dawn-j14mh) and the code source and more examples in this [repo](https://github.com/AbderrahimSoubaiElidrissi/tfjs-examples)

I hope this article gives you a getting started to ML and Tensorflow.js in web applications.

Well in this blog post I will give you simple steps to put machine learning into as frontend developer that has nothing to do with AI or ML before.

Smarter frontend app with Tensorflow.js


Last time I created a simple cli utility to help me to shortener url .
in today's post I will make another cli but this time more interactive to help me to send emails from my terminal 

## What is Deno (in case you don't know what is)

Deno is a simple, modern and secure runtime for JavaScript and TypeScript that uses V8 and is built in Rust. 

The first version is recently [released](https://deno.land/v1)


## Installation Deno

Using shell

```sh
curl -fsSL https://deno.land/x/install/install.sh | sh

```

Using homebrew


```sh
brew install deno

```

## Let's create our CLI

I want to build a interactive CLI (Command line interface) to help me to send emails from my terminal.

First thing to do is asking the user for his email, subject and message to send after this we will send the email using smtp (Google servers) 

### Folder structure

let create a new folders with these files

```
$ mkdir dmailer && cd dmailer
$ mkdir src
$ touch src/mod.ts
$ touch src/deps.ts

```



### Show me the code

The deps.ts file will contains all our dependencies as is [recommended](https://deno.land/manual/linking_to_external_code#it-seems-unwieldy-to-import-urls-everywhere) by deno team  

let open deps.ts and add the following code

```ts
// re-exporting all needed deps
export { default as Ask } from 'https://deno.land/x/ask/mod.ts';

export { SmtpClient } from "https://deno.land/x/smtp/mod.ts";
export { ConnectConfigWithAuthentication } from "https://deno.land/x/smtp/config.ts";

export { red, green, bold } from "https://deno.land/std/fmt/colors.ts";


```

just exporting two dependencies `Ask` (for interactive cli) and `SmtpClient` for creating a SMTP client and some cli console helpers.


Now Open src/mod.ts file and paste the following code
```ts
import { Ask, SmtpClient, ConnectConfigWithAuthentication, red, green, bold } from "./deps.ts";

// create SMTP client

const client = new SmtpClient();

const config: ConnectConfigWithAuthentication = {
  hostname: "smtp.gmail.com",
  port: 465,
  username: "email@gmail.com", // put a valid gmail account and make sure you are able to use this account with smtp
  password: "SuperPass@", // password
}

await client.connectTLS(config);

// create Ask instance 

console.log(bold("*** Welcome to DMailer ***"))


const ask = new Ask({
  prefix: '>'
});

const answers = await ask.prompt([
  {
    name: 'from',
    type: 'input',
    message: 'Enter your Email:'
  },
  {
    name: 'to',
    type: 'input',
    message: 'Enter receiver Email:'
  },
  {
    name: 'subject',
    type: 'input',
    message: 'Enter a Subject:'
  },
  {
    name: 'content',
    type: 'input',
    message: 'Write your message:'
  }
]);


try {

  if (!answers) {
    throw Error("Something is wrong !!")
  }
  const { from, to, subject, content } = answers
  if (!from) {
    throw Error("Your email is required !!")
  }
  if (!to) {
    throw Error("The email is required !!")
  }
  if (!subject) {
    throw Error("The subject is required !!")
  }

  if (!content) {
    throw Error("The message is required !!")
  }

  await client.send({
    from,
    to,
    subject,
    content,
  });

  console.log(green(`Success: Email sent to  ${to}`))


} catch (error) {

  console.log(red(`Error: ${error}`))

}
```

Before sending your email using gmail you have to allow non secure apps to access gmail you can do this by going to your gmail settings [here](https://myaccount.google.com/lesssecureapps).



## Test & Install

to test your code run this in the root folder

```
 deno run --allow-net --allow-read src/mod.ts 

```
The result 

![](media/Screenshot-20200611021956-1005x225.png)

The email :

![](media/Screenshot-20200611022219-751x277.png)


To install your script globally with a specific name:

```
deno install --allow-net --allow-read --name dmail src/mod.ts

```

Now I can use my command like that : 

```
$ dmail

```

## Output : 


![](media/Screenshot-20200611023011-871x250.png)




Hack around deno again and create an interactive mail utility using Deno

Create interactive mail utility CLI Tool using Deno


Deno is the new project from the creator of node.js Ryan Dahl who wants to correct the mistakes previously made on node.js runtime I suggest you  watch this [video](https://www.youtube.com/watch?v=M3BM9TB-8yA&vl=en).

## What is Deno

Deno is a simple, modern and secure runtime for JavaScript and TypeScript that uses V8 and is built in Rust. 

The first version is recently [released](https://deno.land/v1)

## 2. Features

- Secure by default. No file, network, or environment access, unless explicitly enabled.
- Supports TypeScript out of the box.
- Ships only a single executable file.
Has built-in utilities like a dependency inspector (deno info) and a code formatter (deno fmt).
- Has a set of reviewed (audited) standard modules that are guaranteed to work with Deno: deno.land/std

From my little experience, the best way to avoid being stressed in this kind of situations is to be more and more exposed to this thing your brain will find a way to handle this situation

## 3. Installation Deno

Using shell

```sh
curl -fsSL https://deno.land/x/install/install.sh | sh

```

Using homebrew


```sh
brew install deno

```

## 3. Let's build something

I want to build a simple CLI (Command line interface) to help me to shorten a long URL easily.

So basically I will catch the url from CLI arguments, test URL validity and create short URL using [cleanuri](https://cleanuri.com) free API

### Folder structure

let create a new folder with these files

```
MySuperCLI
└── src
    └── main.ts

```

### Show me the code


Open src/main.ts file and paste the following code
```ts
const { args: [url] } = Deno;
import { red, green, blue, bold } from "https://deno.land/std/fmt/colors.ts";

const baseURL: string = "https://cleanuri.com/api/v1/"



function isUrlValid(input: string) {
    const regexQuery = "^(https?://)?(www\\.)?([-a-z0-9]{1,63}\\.)*?[a-z0-9][-a-z0-9]{0,61}[a-z0-9]\\.[a-z]{2,6}(/[-\\w@\\+\\.~#\\?&/=%]*)?$";
    var url = new RegExp(regexQuery, "i");
    return url.test(input);
}

async function shortify(url: string): Promise<{ result_url: string }> {

    if (url === "" || url === undefined) {
        throw { error: "Please provide an url" };

    }

    if (!isUrlValid(url)) {
        throw { error: "Please provide a valid url" };
    }

    const options = {
        method: 'POST', headers: {
            'Content-Type': 'application/json'
        },
        body: JSON.stringify({ url })
    };
    const res = await fetch(`${baseURL}shorten`, options);
    const data = await res.json();
    return data;
}

try {

    const { result_url } = await shortify(url);
    console.log(blue(`${bold('Long URL:')}: ${url}`))
    console.log(green(`${bold('Short URL:')} ${result_url} `))


} catch ({ error }) {

    console.log(red(`Error: ${error}`))

}
```

In deno there is no package.json file so the  modules are imported using a URL and they're cached the first time you run your app.

```ts
const { args: [url] } = Deno;
import { red, green, blue, bold } from "https://deno.land/std/fmt/colors.ts";

```

 In the code above, I import colors function to colorize the output message and I destruct the args variable from the Deno namespace which simply returns the arguments passed to a script and gets the first argument.

 ```ts
function isUrlValid(input: string) {
    const regexQuery = "^(https?://)?(www\\.)?([-a-z0-9]{1,63}\\.)*?[a-z0-9][-a-z0-9]{0,61}[a-z0-9]\\.[a-z]{2,6}(/[-\\w@\\+\\.~#\\?&/=%]*)?$";
    var url = new RegExp(regexQuery, "i");
    return url.test(input);
}

async function shortify(url: string): Promise<{ result_url: string }> {

    if (url === "" || url === undefined) {
        throw { error: "Please provide an url" };

    }

    if (!isUrlValid(url)) {
        throw { error: "Please provide a valid url" };
    }

    const options = {
        method: 'POST', headers: {
            'Content-Type': 'application/json'
        },
        body: JSON.stringify({ url })
    };
    const res = await fetch(`${baseURL}shorten`, options);
    const data = await res.json();
    return data;
}

```

After I create two function the first one to validate url (Stolen from stackoverflow) and the second is just an async function that return the result from API

```ts

try {

    const { result_url } = await shortify(url);
    console.log(blue(`${bold('Long URL:')}: ${url}`))
    console.log(green(`${bold('Short URL:')} ${result_url} `))


} catch ({ error }) {

    console.log(red(`Error: ${error}`))

}

```

the last part it the code execute every time you call CLI from your terminal

## Test & Install

to test your code run this in the root folder

```
 deno run --allow-net src/main.ts https://www.google.com

```

if everything goes right you will see both URLs long and short one 

To install your script globally with a specific name Deno provides a [Script installer](https://deno.land/manual/tools/script_installer) for this :


```
deno install --allow-net --name shortify src/main.ts

```

Now I can use my command like that : 

```
shortify https://www.soubai.me/posts/design-patterns-in-javascript-made-easy-part-iii

```

## Output : 


![](media/Screenshot-20200531004639-880x145.png)


Code available on [github](https://github.com/AbderrahimSoubaiElidrissi/deno-url-shorten-cli)

Today we will build a small CLI to help us to shorten url using Deno

build CLI Tool with Deno


2019 was a very exciting year for me it’s the year when I made my first official public speaking. I was so stressed, my voice was waving and I miss a lot of information per slide..

being in such a stressful situation means that I’m not in my comfort zone anymore, I believe if I want to learn I have to escape from the comfort zone. I learned a lot while trying to survive out of my zone:

## 1.Going deep

It’s easy to memorize some technical words about x or y technology and talk about it (like we do in school projects) especially when your audience is junior, but being a speaker (a good one ) enforce you to know almost everything about what’s you will present (History, Terminology, Best practices…), try to predict your audience questions and consume more resources. This process will give you more self-confidence and help you to learn about this complex stuff, And honestly knowing this shadow part of the subject gives you more ability to understand the “why” for a tool/technology.

## 2. Handling stress

The stress is the daily struggle of everyone but when you’re in a conference speaker you got more because of multiple reasons :

- Miss-understanding something
- Technical issues
- Exceed time without finishing your presentation
- Dyslexia
- broken English
- …

From my little experience, the best way to avoid being stressed in this kind of situations is to be more and more exposed to this thing your brain will find a way to handle this situation

## 3. Network

The biggest thing I think I benefited a lot in expanding my professional and personal network. Make new friends learning other people culture and meet a very influencing people that can help you to get a job, join a Bootcamp…

don’t forget sharing is caring


2019 was a very exciting year for me it’s the year when I made my first official public speaking...

What I learned by Becoming a Conference Speaker


in my last two articles, I talked about design patterns (Creational and Structural) in javascript using typescript. by definition design patterns are the most common solutions for problems in software design.

Today I will finish up this serie of articles with **Behavioral Design Patterns**. According to Wikipedia behavioral design patterns are design patterns that identify common communication patterns among objects and realize these patterns. We will go through three design patterns in this article :

## 1.Observer

Observer pattern is used when there is “one to many relationship” between objects such as if one object is modified, it’s dependent objects are to be notified automatically.

**Problem** :

Imagine that we have a weather system with a central dashboard and multiple sensors. Every time the sensor gets a new value it should appear on the dashboard. The technical solution, if I don’t use a design pattern, the dashboard will be -every time- (Loops every 5 sec for example) checking for the new value from the sensor, this solution is limited and it’s very costly especially when we have a lot of sensors to check every time.

**Example** (Using design pattern) :

```ts
class Observer {
  constructor() {
    this.observers = [];
  }

  subscribe(fn) {
    this.observers.push(fn);
  }

  unsubscribe(fn) {
    this.observers = this.observers.filter(subscriber => subscriber !== fn);
  }

  broadcast(data) {
    this.observers.forEach(subscriber => subscriber(data));
  }
}
class Observer {
  constructor() {
    this.observers = [];
  }

  subscribe(fn) {
    this.observers.push(fn);
  }

  unsubscribe(fn) {
    this.observers = this.observers.filter(subscriber => subscriber !== fn);
  }

  broadcast(data) {
    this.observers.forEach(subscriber => subscriber(data));
  }
}

// example
const stationObserver = new Observer();

stationObserver.subscribe(value => {
  console.log(value);
});

let i = 0;
setInterval(() => stationObserver.broadcast(i++), 1000);

const stationObserver = new Observer();

stationObserver.subscribe(value => {
  console.log(value);
});

let i = 0;
setInterval(() => blogObserver.broadcast(i++), 1000);
```

Our next design pattern this night is :

## 2.Interpreter

The interpreter pattern is used to define a grammatical representation for a language and provide an interpreter to deal with this grammar. This pattern involves implementing an expression interface which tells to interpret a particular context.

**Problem** :

Imagine you have a system that helps people with Amyotrophic lateral sclerosis (Like Steven Hawken) to communicate with the world, your program /device can get the data from the “brain” of the patient and convert it to English. One day you want to extend your system to cover more languages: Arabic, French if you don’t use a design pattern you will find yourself repeating your code for every language you want to add.

**Example** (Using design pattern) :

```ts
interface IInterpretable {
  interpret(): string;
}

class GreetingEnglishService implements IInterpretable {
  interpret() {
    return "Hi";
  }
}

class GreetingFrenchService implements IInterpretable {
  interpret() {
    return "Salut";
  }
}

//Example

let greeter = null;
let userLangague = "Fr";

if (userLangague === "Fr") {
  greeter = new GreetingFrenchService();
} else {
  greeter = new GreetingEnglishService();
}

console.log(greeter.interpret());
```

Finally, ladies and gentlemen :

3.**Memento :**

The memento design pattern is used to restore the state of an object to a previous state. Generally, it used when we need to restore an object back to its previous state (e.g. “undo” or “rollback” operations).In the Memento pattern, a memento object will hold the state of another object.

**Problem** :

We are creating our super program that can handle the git commit history everything works correctly but we need to make some “time travel” (state manipulation ).

**Example** (Using design pattern):

The implementation here is with three objects: the _originator_, a _caretaker,_ and a _memento_. The originator is an object that has an internal state. The caretaker is going to do something to the originator but wants to be able to undo the change. The caretaker first asks the originator for a memento object. Then it does whatever operation (or sequence of operations) it was going to do. To roll back to the state before the operations, it returns the memento object to the originator. (Wikipedia)

```ts
class Memento {
  private state: Object;

  public constructor(state: Object) {
    this.state = state;
  }

  public getState(): Object {
    return this.state;
  }
}

class Originator {
  private state: Object;

  public setState(state: Object) {
    console.log("Originator: Setting state to ", this.state);

    this.state = state;
  }

  public commit(): Memento {
    console.log("Originator: Saving to Memento.");

    return new Memento(this.state);
  }
  public roolback(m: Memento) {
    this.state = m.getState();

    console.log("Originator: State after restoring from Memento: ", this.state);
  }
}

class Caretaker {
  private mementos = [];

  public addMemento(m: Memento) {
    this.mementos.push(m);
  }

  public getMemento(index): Memento {
    return this.mementos[index];
  }
}

// Example

let caretaker: Caretaker = new Caretaker();
let originator: Originator = new Originator();
originator.setState({ id: Math.random(), message: "initial commit" });
originator.setState({ id: Math.random(), message: "Tiny fix" });
caretaker.addMemento(originator.commit());
originator.setState({ id: Math.random(), message: "Add cors" });
caretaker.addMemento(originator.commit());

originator.setState({ id: Math.random(), message: "Hopla" });

originator.roolback(caretaker.getMemento(0));
```


in my last two articles, I talked about design patterns (Creational and Structural) in javascript using typescript...

Design patterns in Typescript made easy — part III


Javascript is the future for a lot of reasons:

1.  One single language for the frontend and backend
2.  Simple learning curve
3.  Great community (packages managers / frameworks / testing tools)

In light of this great development, other technologies related to javascript are born one of them is the node.js according to Wikipedia :

> Node.js is an open-source, cross-platform JavaScript run-time environment that executes JavaScript code outside of a browser.

In simple words, node.js is the technology which allows us to run javascript in server side to make a backend applications such API, Workers …

You can download it from the official website for any OS :

[https://nodejs.org/en/download](https://nodejs.org/en/download/)

let’s have our demo app :

Create _index.js_

```js
console.log("Hello Medium");
```

Save the file and run it using this command :

```sh
node index.js
```

![](media/1_eUuXPB_SYI0qRDRsiyczBw.png)

Here we launched our app as a single process and after execution, it shut down but in real cases (API server for example) we need to maintain our application executed So we use tools called process managers such _forever_, _strongloop_, _PM2_ to easily run, maintain, log…;

Today, I will use [PM2](http://pm2.keymetrics.io/) :

Let’s install it first :

```
npm install pm2 -g
```

Starting our application with a watch flag :

```
pm2 start index.js --watch
```

![](media/1_zak8W2ZYSuf-wQjfp15aOw.png)

After that we preview the log file :

```
pm2 logs
```

![](media/1_Xjo9MJ2AWUiEYIdzsTKTYQ.png)

Wow! Cool our application is re-launched every time (watched by PM2).

Returning to figure 2 we see all the processes launched by PM2 (single instance <=> single CPU).

Imagine our application is the next _“1B dollar startup”_ API, this kind of applications should have a scalable behavior for fault tolerance and high availability.

To achieve this type of architectures we have a lot of choices: **Serverless** architecture, **Microservices** architecture, **Could based solutions** (AWS Load Balancer )..; In my example, I will use PM2 again to solve the problem,

I will run the applications in all available CPUs using PM2 **cluster mode :**

> cluster mode **:** allows networked Node.js applications to be scaled across all CPUs available, **without any code modifications**.

Let’s define JSON file :

_app.json_

```json
{
  "apps": [
    {
      "script": "index.js",
      "instances": "max",
      "exec_mode": "cluster"
    }
  ]
}
```

Let’s start the cluster :

```
pm2 start app.json

```

![](media/1_YGBhlZ5OlsIoCtMfUM4cxg.png)

Yes, my machine has 4 available CPUs and PM2 has used all of them to serve the application with the same shared port, this simple edit in the execution mode give our application more performance and reliability

For more information check PM2 [documentation](http://pm2.keymetrics.io/docs/usage/cluster-mode/#cluster-mode).


Node.js is an open-source, cross-platform JavaScript run-time environment that executes JavaScript code outside of a browser...

Run your node.js app on the cluster mode


Today we will talk about structural design patterns which are considered as design patterns that make easy/possible relations between different classes.

But first I have to talk about an important design pattern that belongs to our last articles’s design patterns category (creational), the Factory design pattern is defined as the solution of creating an object of a class without exposing the creation logic to the client and refer to a newly created object using a common interface.

**The problem:**

Imagine a scenario where there are multiple cases in the client class and we call multiple new keywords for creating new objects.

```ts
if (value == "Ferari") {
  let car = new Ferari();
} else if (value == "Bugati") {
  let car = new Bugati();
}
```

**The solution:**

The Factory method pattern is to define an interface for creating objects but delegates the object creation to the subclasses.

```ts

interface CarFactory
{
    public function makeCar(type:string);
}

interface Car
{
    public function getType();
}

class Ferari inmplements Car
{
     public function getType()
     {
        return 'Ferari';
     }

}

class Bugati inmplements Car
{
     public function getType()
     {
        return 'Bugati';
     }

}


class CarFactory implements CarFactory
{
    public function makeCar()
    {
      if(value == 'Ferari'){
          let car = new Ferari();
      }
     else if(value == 'Bugati'){
        let car = new Bugati();
     }

    }
}

```

```ts
/* Client */

let factory = new CarFactory();

let car = factory.makeCar("Ferari");

console.log(car.getType());
```

## 1.Adapter design pattern

[](media/0_cXCec3AXy8R3cqlz.png)

The adapter is a structural design pattern that allows objects with incompatible interfaces to collaborate just like HDMI <-> DVI adapter

**The problem :**

Imagine that you have an app that works with data in XML format, but at some point, you need to use a library that can only work with JSON.

For example our application use a soap API (XML output)to get weather data for the north of the country from a provider and show it in LCD display the code works just fine but you decide to use another API to get the data of the south of the country and here you have to use a Rest API (JSON output) you have two solutions to deal with the new situation : 1- to make a new class to deal with the rest API OR 2 — to use Adapter design patterns.

**The solution :**

We have adapter here that can communicate with the client this adapter can understand both types of returns (XML and JSON) and provide a data for our LCD display (client)

```ts
interface IDisplayable {
  getData();
  display();
}

class SOAPService implements IDisplayable {
  getData(): void {
    //call api and get xml data
    return xmlData;
  }
  display(data): void {
    console.log(this.convertFromXML(data));
  }

  convertFromXML(): string {
    //convert from XML to displayable format
    return string;
  }
}

class JSONService implements IDisplayable {
  getData(): void {
    //call api and get json data
    return jsonData;
  }

  display(data): void {
    console.log(this.convertFromJSON(data));
  }

  convertFromJSON(): string {
    //convert from JSON to displayable format
    return string;
  }
}

class APIAdapter implements IDisplayable {
  soapService: SOAPService = new SOAPService();
  jsonService: JSONService = new JSONService();
  type: string = "";

  constructor(type: string) {
    this.type = type;
  }

  getData(): void {
    if (type === "soap") {
      return this.soapService.getData();
    } else if (type === "json") {
      return this.jsonService.getData();
    } else {
      throw new Error("Format not handled ");
    }
  }

  display(): void {
    if (type === "soap") {
      this.soapService.display();
    } else if (type === "json") {
      this.jsonService.display();
    } else {
      throw new Error("Format not handled ");
    }
  }
}
```

We have adapter here that can communicate with the client this adapter can understand both types of returns (XML and JSON) and provide a data for our LCD display (client)

## 2.Facade design pattern

[](media/0_MbRd9Br5rRuIMlvt.png)

Facade pattern hides the complexities of the system and provides an interface which the client can access the system. This type of design pattern adds an interface to the existing system to hide its complexities.

**The problem:**

For example, we have a class called by our client to upload the image to Amazon S3 but before upload, we have to check, convert, rename the file. Every action here is related to a specific class we have a class to check the file and another to rename it using a specific naming strategy and checking file using a machine learning predict library to prevent a pornographic content.

As we see here we have a complex system to upload a file we will use a Facade to reduce all this complexity in a single method (one call)

**The solution :**

// call example**import** Uploader = Facade.Upload;  
// checking , renaming , convert and upload the file let file = new File('./pixel.png');  
Uploader.upload(file);

Here the facade method ‘upload’ provides to the client an easy way to reduce all the process with a simple one call to a static method

```ts
class Uploader {
  uploadToS3(file) {
    /* doing some AWS SDK things here */
  }
}

class NameChanger {
  rename(file) {
    /* doing some Naming here */
  }
}

class Converter {
  convert(file) {
    /* doing some Converting here */
  }
}

class Checker {
  predict(file) {
    /* doing some Machine Learning  here    */
  }
}

namespace Facades {
  export class Upload {
    static Upload(file) {
      let nameChanger: NameChanger = new NameChanger();
      let converter: Converter = new Converter();
      let checker: Checker = new Checker();
      let uploader: Uploader = new Uploader();

      let myFile = checker.predict(file);

      myFile = nameChanger.rename(myFile);

      myFile = converter.convert(myFile);

      uploader.uploadToS3(myFile);
    }
  }
}
```

```ts
// call example
import Uploader = Facade.Upload;
// checking , renaming , convert and  upload the file
let file = new File("./pixel.png");
Uploader.upload(file);
```

Here the facade method ‘upload’ provides to the client an easy way to reduce all the process with a simple one call to a static method

## 3.Proxy design pattern

[](media/0_ahKhoqRQSx9OdZ4s.png)

Proxy is a structural design pattern that works as an interface of another class, instead of calling a function from a real class we go through the proxy

**The problem :**

We have a powerful object that logs actions in our database for example but to log in the database you should have a role for that (permissions), there is multiple solutions for this problem but the best way is to use Proxy.

In an ideal world, we would put this code directly into the object’s class, but that is not always possible. For instance, the class may reside in a closed 3rd-party library.

**The solution :**

```ts
interface ILogable {
  log(action): void;
}

class Logger implements ILogable {
  log(action) {
    /* doing some Log action here */
  }
}

class LoggerProxy implements ILogable {
  constructor() {
    this.checkAccess();
  }

  log(action) {
    let logger = new Logger();

    logger.log(action);
  }

  checkAccess() {
    /* check that the user have a correct role to Log in the database */
  }
}
```

```ts
// call example
let logger = new LoggerProxy(); // here we check for permissions// later we loglogger.log('Delete payment resource ID 123');
```

First, we create an interface to make class and proxy interchangeable we invoke the original class function into the proxy function and before instantiation of the proxy we check the permission of the connected user.

Part I : [https://soubai.me/posts/design-patterns-in-javascript-made-easy-part-i](https://soubai.me/posts/design-patterns-in-javascript-made-easy-part-i)


Today we will talk about structural design patterns which are considered as design patterns that make easy/possible relations between…

Design patterns in Typescript made easy — part II


> If you’re a Here probably you heard about Design patterns Or You are Javascript enthusiast in both cases Welcome. Today I will write about…

A pattern is a reusable solution that can be applied to commonly occurring problems in software design — in our case — in writing Javascript web applications. Another way of looking at patterns are as templates for how we solve problems — ones which can be used in quite a few different situations. _(Book: Learning JavaScript Design Patterns by Adnan Osmani)_

Before we start I have to configure the Javascript project to use Typescript to benefit from the power of Oriented Object Programming such interfaces and types :

First thing to do is to grab my boilerplate from github :

```
$ git clone https://github.com/AbderrahimSoubaiElidrissi/nodejs-typescript-boilerplate.git
$ cd nodejs-typescript-boilerplate\
$ npm install


```

now we have our work space ready we can start with the first design pattern

## 1.Constructor Pattern :

> In classical object-oriented programming languages, a constructor is a special method used to initialize a newly created object once memory has been allocated for it. In JavaScript, as almost everything is an object, we’re most often interested in _object_ constructors.

**Problem :**

Our application uses an object to structure the data and call service’s methods so we need to initialize those objects and populate there properties.

**Solution :**

The Constructor is a “Magic” method used to initialize the the properties of an object and for invoking methods on the object initialization.

**Example :**

got to the src/app.ts edit file with this snippet :

```ts
class Greeter {
  greeting: string;

  constructor(message: string) {
    this.greeting = message;
    console.log(`Hello ${this.greeting} !`);
  }
}

let greeter = new Greeter("world"); // Hello world !
```

We can see in the console “Hello world” after ‘npm start’ which means our constructor has set the value passed in the parameters to the property greeting

## 2.Singleton Pattern :

![](media/1_FFhIuK-Y9k1FrBXLWKtYyg.png)

The next pattern is one of the most famous patterns because it’s a member of squad Gangs of Four Design patterns .

**Problem :**

Our application needs only one single instance of a given class with a global public access (example Session instance).

**Solution :**

Singleton pattern is a solution to restrict the instantiation of a class and ensures that only one instance of the class exists. Singleton most provide a global access point to get the instance of the class generally we use the singleton patterns to achieves functions like Logging ,Caching,Session…

**Example** :

```ts
class Singleton {
  private static instance: Singleton = new Singleton();

  constructor() {
    if (Singleton.instance) {
      throw new Error(
        "Error: Use SingletonClass.getInstance() instead of new."
      );
    }
    Singleton.instance = this;
  }

  public static getInstance(): Singleton {
    return Singleton.instance;
  }
}
```

The following code will throw the error because we have early an instance of Singleton class stored in the property instance. Another interesting Idea to enforce the Singleton patterns without testing instance in the constructor is to make the constructor of the singleton class private this will prevent people from instantiate your class

```ts
let mySuperInstance = new Singleton(); // Error
```

To instantiate the singleton you have to call the static method getInstance()

```ts
let mySuperInstance = Singleton.getInstance(); // success
```

## 3.Prototype pattern :

![](media/1_zZ5_U5KmjuSw5wsbp9GMGw.png)

Our next design pattern is the Prototype design patterns. Prototype is used when the type of objects to create is determined by a prototype generally to improve the performance of the architecture by reducing the cost of creating Objects.

**Problem :**

Our applications need objects to achieve function but the creation of the object (using operator new) is costly in the point of view performances for example a creation of an instance of an object involves calling the database or making an API calls .

**Solution :**

Prototype patterns is required when object creation is time consuming, and costly operation, so we create an object with the existing object itself. One of the best available way to create an object from existing objects are **clone() method**. Clone is the simplest approach to implement the prototype pattern. However, it is your call to decide how to copy existing object based on your business model.

**Example :**

```ts
class Singleton {
  private static instance: Singleton = new Singleton();

  constructor() {
    if (Singleton.instance) {
      throw new Error(
        "Error: Use SingletonClass.getInstance() instead of new."
      );
    }
    Singleton.instance = this;
  }

  public static getInstance(): Singleton {
    return Singleton.instance;
  }
}
```

I create the interface ICloneable to enforce the prototype class to implement the method clone(). The clone method return an exact clone of the instance of prototype class in result we have two instances of a class with only one use of the “new” operator and this practice reduces the cost of creating a heavy object (Loops, database calls …)

To test the design pattern :

```ts
//the instance
let prototype = new Prototype(1);//the clone
let pClone = prototype.clone();console.log(prototype.list\[10\] === pClone.list\[10\] ); //true
console.log(prototype.identity === pClone.identity); //true

```


A pattern is a reusable solution that can be applied to commonly occurring problems in software design

Design patterns in Typescript made easy — part I



> Voice-activated applications used to be the stuff that science fiction was made of, but not anymore. Today, these applications are readily used on a daily basis by adults and children alike for both business and personal uses. — unknown

Everyday the world tends to become an amazing place due to the science and technologies’s big progress.this progress is driven by huge amount of tools that we use everyday .

One of the best example of those amazing things are the personal assistant apps, these virtual creatures are really astonishing and can do many things to help you either in personal and professional daily life, the major advantage of this kind of applications is their ability of voice interaction with the
user.

The voice driven applications are then the best solution to accomplish your daily mechanical tasks due to their high performance and
intuitiveness.

Here is a great video of Google Assistant Demo in Google I/O 2018 :

<iframe width="560" height="315" src="https://www.youtube.com/embed/d40jgFZ5hXk"
frameborder="0"
allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen></iframe>

Google Assistant can Also helps you to :

- Make calls /messaging
- Google something very quickly (search for near restaurant / routes / pharmacies).
- Set alarms
- Check the weather
- Describe your personality
- Tell you a jokes
- …

So Today I decide to play around the speech recognition and see how can I hack something :) for the web.

I will create a remote control application to control a video player using voice in an HTML page.

The Javascript API for [speech recognition](https://w3c.github.io/speech-api/speechapi.html) makes the voice manipulation easier for web pages. The only requirement for that and especially when using google chrome browser is the web page should be served in a secure protocol (HTTPs).

So the first thing we have to do is to create a secure localhost using node.js

```
//make the project directory
mkdir remote-control-with-speech-recognition && cd remote-control-with-speech-recognition
// initialize the npm
npm init
// install required package
npm install https express

```

After that we have to generate the _privatekey.pem_ and _certificate.pem_ files using the following commands :

```
openssl genrsa -out privatekey.pem 1024
openssl req -new -key privatekey.pem -out certrequest.csr
openssl x509 -req -in certrequest.csr -signkey privatekey.pem -out certificate.pem
```

Creating the server file :

```
// create a server.js file
touch server.js
```

```javascript
const fs = require("fs");
const https = require("https");
const express = require("express");

const app = express();
app.use(express.static(__dirname + "/public"));

const privateKey = fs.readFileSync("privatekey.pem").toString();
const certificate = fs.readFileSync("certificate.pem").toString();

const httpOptions = { key: privateKey, cert: certificate };
https.createServer(httpOptions, app).listen(8000, () => {
  console.log(">> Serving on " + 8000);
});
app.get("/", function(req, res) {
  res.sendFile(__dirname + "/public/index.html");
});
```

Creating index.html in public directory :

```
// create a public/index.html file
mkdir public && cd public && touch index.html

```

```html
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <title>Speech recognition video control</title>
    <meta name="viewport" content="width=device-width, initial-scale=1" />
  </head>
  <body>
    <video
      id="video"
      controls=""
      preload="none"
      muted="”muted”"
      poster="https://media.w3.org/2010/05/sintel/poster.png"
    >
      <source
        id="mp4"
        src="https://media.w3.org/2010/05/sintel/trailer.mp4"
        type="video/mp4"
      />
      <source
        id="webm"
        src="https://media.w3.org/2010/05/sintel/trailer.webm"
        type="video/webm"
      />
      <source
        id="ogv"
        src="https://media.w3.org/2010/05/sintel/trailer.ogv"
        type="video/ogg"
      />
      <p>Your user agent does not support the HTML5 Video element.</p>
    </video>
    <p>
      Say : Play / Stop / Mute / Unmute
    </p>
    <script src="RemoteControl.js"></script>
    <script src="main.js"></script>
  </body>
</html>
```

Creating main.js and RemoteControl.js files in public directory :

```
// create a public/main.js file
 touch main.js
```

```js
var video = document.querySelector("#video");

var remoteControl = new RemoteControl(video);
var recognition = new webkitSpeechRecognition();
recognition.continuous = true;
recognition.interimResults = true;
recognition.lang = "en-US";
recognition.continuous = true;
recognition.start();

recognition.onresult = function(event) {
  for (var i = event.resultIndex; i < event.results.length; ++i) {
    if (event.results[i].isFinal) {
      if (event.results[i][0].transcript.trim() == "play") {
        remoteControl.play();
      }
      if (event.results[i][0].transcript.trim() == "stop") {
        remoteControl.stop();
      }
      if (event.results[i][0].transcript.trim() == "mute") {
        remoteControl.mute();
      }
      if (event.results[i][0].transcript.trim() == "unmute") {
        remoteControl.unmute();
      }
      console.info(`You said : ${event.results[i][0].transcript}`);
    }
  }
};
```

```
// create a public/RemoteControl.js file
touch RemoteControl.js
```

```js
class RemoteControl {
  constructor(element) {
    this.videoElement = element;
  }

  play() {
    this.videoElement.play();
  }
  stop() {
    this.videoElement.pause();
    this.videoElement.currentTime = 0;
  }
  mute() {
    this.videoElement.muted = true;
  }
  unmute() {
    this.videoElement.muted = false;
  }
}
```

Save your work and then run the app

```
node server.js

```

Goto https://localhost:8000

You will see this :

![s1](media/1_h3YboNd4YmvJzOFnV54VzQ.jpeg)

Click on advanced :

![s2](media/1_74wB1up3KDuYJv6DBZAZbQ.png)

And then Proceed to localhost (unsafe)

![s3](media/1_Bksq-xaS0ByW6d9qlSk4aw.png)

Here we go !you can control this video using your voice try to Say :play/stop/mute/unmute

You can also grab the code from [Github](https://github.com/AbderrahimSoubaiElidrissi/remote-control-with-speech-recognition) ..

Happy hacking!


Voice-activated applications used to be the stuff that science fiction was made of, but not anymore. Today, these applications are readily used on a daily basis by adults and children alike for both business and personal uses...

Getting started with Voice Driven Applications using Node.js


NodeJS is a great technology for web/IoT projects it’s fast , resources friendly and beautiful. Also among its qualities there is code re-usability and we should says thank to NPM.

Node package manager (NPM) is a great tool for every JavaScript developer it’s provides a million of packages that can be used in your projects.

Today I want to share a simple tutorial on how to get started with face detecting in nodejs. Hence i’ll be using a nodejs package named face-recognition created and maintained by the awesome Vincent Mühler the package provide

> Simple Node.js API for robust face detection and face recognition. This a Node.js wrapper library for the face detection and face recognition tools implemented in dlib.

Required libs for Linux and OSX :

```
cmake (sudo apt-get install cmake)

libx11 (XQuartz on OSX) for the dlib GUI (sudo apt-get install libx11-dev)

libpng for reading images (sudo apt-get install libpng-dev)
```

Let’s have fun :

```
npm install face-recognition
```

create new file index.js and open the project in your favorite code editor (mine is VS Code)

```
touch index.js && code .

```

past this code on your index.js

```javascript
const fr = require("face-recognition");
// load image from local
const image = fr.loadImage("image.jpg");
// intialize the window to showup your image
const win = new fr.ImageWindow();
// intialize face detector
const detector = new fr.FrontalFaceDetector();
// get the upscaled version of the supplied image
const sacaledimg = fr.pyramidUp(image);
// detect faces and returns all rects object (faces dimensions)
const gotFaceRects = detector.detect(sacaledimg);
// supply image to the window
win.setImage(image);
// draw rectangles over the image
const f = 0.5;
const faceRects = gotFaceRects.map(
  rect =>
    new fr.Rect(rect.left * f, rect.top * f, rect.right * f, rect.bottom * f)
);
faceRects.forEach(r => {
  win.addOverlay(r);
});
// pause the program until you hit Enter
fr.hitEnterToContinue();
```

Results :

```
// in your terminal run
node index.js
```

before
![type-through-time.jpg](media/1_uRUhqr7qxE8kaNZ9TpHWbw.png)

After
![type-through-time.jpg](media/1_YXA8ViCUA8E026fe3Rke4w.png)


NodeJS is a great technology for web/IoT projects it’s fast , resources friendly and beautiful. Also among its qualities there is code re-usability and we should says thank to NPM...

July 11, 2022cloud
Breaking down etcd

May 15, 2022devops
Look mom no TravisCI

May 14, 2021javascript
Build your own webpack plugin